-- Logs begin at Mon 2022-07-18 09:41:51 UTC, end at Mon 2022-07-18 11:05:17 UTC. -- Jul 18 10:23:37.297361 dv-cinder-849498-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 10:23:37.311065 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 10:23:37 2022] *** Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: nodename: dv-cinder-849498-1-iscsi Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: machine: x86_64 Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: clock source: unix Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: pcre jit disabled Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: detected number of CPU cores: 4 Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: current working directory: / Jul 18 10:23:37.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: detected binary path: /usr/bin/uwsgi-core Jul 18 10:23:37.314081 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: your processes number limit is 47680 Jul 18 10:23:37.314081 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: your memory page size is 4096 bytes Jul 18 10:23:37.314081 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: detected max file descriptor number: 2048 Jul 18 10:23:37.314081 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: lock engine: pthread robust mutexes Jul 18 10:23:37.314081 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: thunder lock: enabled Jul 18 10:23:37.314081 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 10:23:37.314081 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 10:23:37.354155 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: Python main interpreter initialized at 0x55f36ec029c0 Jul 18 10:23:37.354155 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: python threads support enabled Jul 18 10:23:37.354155 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: your server socket listen backlog is limited to 100 connections Jul 18 10:23:37.354155 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: your mercy for graceful operations on workers is 90 seconds Jul 18 10:23:37.354576 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 10:23:37.354576 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: *** Operational MODE: preforking *** Jul 18 10:23:37.354872 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: *** uWSGI is running in multiple interpreter mode *** Jul 18 10:23:37.354872 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: spawned uWSGI master process (pid: 80380) Jul 18 10:23:37.355086 dv-cinder-849498-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 10:23:37.356127 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: spawned uWSGI worker 1 (pid: 80394, cores: 1) Jul 18 10:23:37.356127 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: spawned uWSGI worker 2 (pid: 80395, cores: 1) Jul 18 10:23:37.356369 dv-cinder-849498-1-iscsi devstack@keystone.service[80380]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 10:23:42.999609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:23:42.999609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes Jul 18 10:23:43.000398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:23:43.000398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes Jul 18 10:23:43.001054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:23:43.001054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes Jul 18 10:23:43.001524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:23:43.001524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes Jul 18 10:23:43.051358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 10:23:43.051643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 10:23:43.051764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] command line args: [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 10:23:43.052015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 10:23:43.052182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 10:23:43.052309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ================================================================================ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 10:23:43.052436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 10:23:43.052436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] command line args: [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 10:23:43.052679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.052679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_dir = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.052896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 10:23:43.052896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ================================================================================ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 10:23:43.053045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_source = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] debug = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_dir = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_source = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.053937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] debug = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.054937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_config_append = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_dir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_config_append = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.055855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_dir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.056508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.056596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.056772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.056902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.056976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.057963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.058949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.059079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.059197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.059283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.059452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.059715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.059887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] publish_errors = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] publish_errors = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.060979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] standard_threads = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] standard_threads = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.061919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_journal = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_json = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_stderr = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_journal = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_syslog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_json = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.062953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_stderr = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.063023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_syslog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.063101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.063209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:23:43.063287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.063488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.063604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.063718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.063853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.063988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.064118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.064248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.064369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.064452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.064532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.064752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.064965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.065043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.065399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.065479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.065655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.065792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.065877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.066916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.067076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.067162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.067337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.067420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.067637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.067807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.067897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.068094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.068211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.068354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.068620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.068694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.068929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.069966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.070988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.071942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.072967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073463 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.073936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.074978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.075135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.075222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.075377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.075497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.075630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.075744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.075827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.076894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077256 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.077971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.078970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.079986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.080980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.081139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.081219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.081389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.081519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.081641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.081819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.081906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.082873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.083962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.084109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.084189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.084349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.084428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.084598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.084732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.084935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085392 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.085968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.086983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.087104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.087225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.087332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.087489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.087630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.087762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.087886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.external = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.088976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.password = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.token = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.089908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.external = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.090982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.password = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.token = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.091939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.092892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.093893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.094997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.095924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.096081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.096199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.096316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.096480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.096604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.096728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.096873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.097857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.098020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.098088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.098262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.098336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.098514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.098614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.098761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.099413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.099526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.099651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.099844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.099924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.100100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.100171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.100336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.100468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.100576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.100834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.100936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.101107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.101186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.101341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.101576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.101671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.101808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.101887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.102893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.103063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.103151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.103326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.103408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.103565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.103645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.103802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.104873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105393 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.105943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.106117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.106196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.106368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.106532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.106618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.106848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.106957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.107099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.107222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.107337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.107467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.107579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.107819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.107895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.108061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.108271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.108349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.108534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.108637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.108794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.108941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.109023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.109277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.109499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.109577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.109757 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.109857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.109986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.110978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.111985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.112146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.112225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.112382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.112460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.112629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.112764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.112982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.113870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.114028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.114116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.114262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.114340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.115139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.115259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.115325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.115537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.115636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.115810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.115938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116572 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.116956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.117902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.118874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119817 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.119969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.120956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.121992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.122155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.122236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.122390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.122468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.122675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.123878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.124049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.124126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.124290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.124370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.124531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.124610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.124805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.130855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.130855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.131958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.132973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133780 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.133943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.134929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.135961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136538 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.136945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.137944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.138863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.139934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.140860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.141998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.142150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.142233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 10:23:43.142305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.142527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.142770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.143015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.143254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.143494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.143793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.144034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.144285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.144556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.144845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.145091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.145345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.145584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.145887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.146618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.146862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.147104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:23:43.147275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 10:23:43.173762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.173964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.174207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.174437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.174676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.174898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.183764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.183965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.212939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.213183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.213285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.213352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.360238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.360666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.410347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.410581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.412525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.412867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.476785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.477981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.478148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.478374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.478591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.503922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504668 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.504937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.505062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.505158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.505374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.505901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.516679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.517052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.530331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.530562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.530790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.531012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.531226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.531363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.531452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.531603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.531705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.531826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.532048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.532264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.532478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.532692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:23:43.549731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.549918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.550014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:23:43.550094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:23:43.550274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.550404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.550529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:23:43.550661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:23:43.550787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.550911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.551041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.551170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.551299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.551444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.551578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.551706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.562946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.563106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.563230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.563339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.563501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.563585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.563783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.563868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.572544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.573260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.575868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.576153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.576438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.576537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.576861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.576945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.577150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.577238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.577415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.577555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.577689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.577845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.577969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.578115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.578277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.578415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.578561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.578682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.578830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.578933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.579184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.579451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.606671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.608317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.608601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.611608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.613338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.613672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.613935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.614216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.614521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.619365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.619665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.619955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.621917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.622212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.626879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.627165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.627429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.627517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.627709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.627797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.627983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.628513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.628800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.629068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.633458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.633752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.634038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.634321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.634596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.634919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.635126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.635395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.649119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.649412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.663408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.663537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.663537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.663672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.663672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.663787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.663864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.663930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.664427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.666768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.669182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.670626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.670890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.671172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.671454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.671749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.672036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.672892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.676719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.677240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.677549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.677833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.678242 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.679912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.691278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.691683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.691956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.692225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.698583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.699015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.701634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.701634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.701863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.702787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.703066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.706140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.706429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.706720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.707010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.707288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.707718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.710506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.710781 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.713859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.714254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.714599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.714890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.715172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.721412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.721704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.721995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.722278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.722561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.722836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.723111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.728979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.729272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.729557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.729835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.730113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.730395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.730673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.743213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.746554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.750065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.750983 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.751680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.753629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.753908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.754190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.754590 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.758494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.760124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.761819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.762097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.762258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.762380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.763995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.767261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.767558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.767839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.768121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.769898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.772149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.775514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.775810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.776097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.776381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.778379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.778683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.779022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.785703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.785999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.786276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.787199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.787199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.787506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.787784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.789259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.789259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.789259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.789259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.789259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.789593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.789685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.791143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.791143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.791143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.794283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:23:43.794571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.794852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.795128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.795409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.795691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.795974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.796238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.796505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.796900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.797081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.797357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.797617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.797880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:23:43.829747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f36ec029c0 pid: 80395 (default app) Jul 18 10:23:43.833364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ef646-ee5c-4169-8d29-4bcfb8d00844 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.834155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ef646-ee5c-4169-8d29-4bcfb8d00844 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.834446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ef646-ee5c-4169-8d29-4bcfb8d00844 None None] PATH_INFO: `/v3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.837250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f36ec029c0 pid: 80394 (default app) Jul 18 10:23:43.838881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Mon Jul 18 10:23:43 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:52.776192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250e1c6-955a-4ba3-b2a0-dbed0afbe49f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:52.776762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250e1c6-955a-4ba3-b2a0-dbed0afbe49f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:52.776762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250e1c6-955a-4ba3-b2a0-dbed0afbe49f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:52.781139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:23:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:52.786878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:52.787314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:52.787412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:52.857596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: INFO dbcounter [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] Registered counter for database keystone Jul 18 10:23:52.879788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG oslo_db.sqlalchemy.engines [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 10:23:52.887913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writer thread running {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 10:23:53.068332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80395) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 10:23:53.068664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80395) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 10:23:53.124139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:53.125797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:53.250322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 10:23:53.411107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250e1c6-955a-4ba3-b2a0-dbed0afbe49f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.481478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: INFO dbcounter [None req-8250e1c6-955a-4ba3-b2a0-dbed0afbe49f None None] Registered counter for database keystone Jul 18 10:23:53.500691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG oslo_db.sqlalchemy.engines [None req-8250e1c6-955a-4ba3-b2a0-dbed0afbe49f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80394) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 10:23:53.507720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writer thread running {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 10:23:53.627741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250e1c6-955a-4ba3-b2a0-dbed0afbe49f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:53.635066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340bdd6-1cf5-407f-9652-24edcc75eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:53.635733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340bdd6-1cf5-407f-9652-24edcc75eff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.636022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340bdd6-1cf5-407f-9652-24edcc75eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.636303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340bdd6-1cf5-407f-9652-24edcc75eff6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.644888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340bdd6-1cf5-407f-9652-24edcc75eff6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:53.698274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4340bdd6-1cf5-407f-9652-24edcc75eff6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 10:23:53.698619 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340bdd6-1cf5-407f-9652-24edcc75eff6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:53.709870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Jul 18 10:23:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:23:53.715960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.730750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e8991-bf47-44c8-a675-ee7f6d66f64e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:53.735295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffae02-230d-4b15-a8d2-9b825fd49264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:53.736428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffae02-230d-4b15-a8d2-9b825fd49264 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.736718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffae02-230d-4b15-a8d2-9b825fd49264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.737026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffae02-230d-4b15-a8d2-9b825fd49264 admin admin] PATH_INFO: `/v3/projects/233b60ec452b417c902a9645e05969db` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.741109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffae02-230d-4b15-a8d2-9b825fd49264 admin admin] RBAC: Authorizing `identity:get_project(project_id=233b60ec452b417c902a9645e05969db)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:53.792071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING keystone.common.rbac_enforcer.enforcer [None req-baffae02-230d-4b15-a8d2-9b825fd49264 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 10:23:53.792694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffae02-230d-4b15-a8d2-9b825fd49264 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:53.797089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Mon Jul 18 10:23:53 2022] GET /identity/v3/projects/233b60ec452b417c902a9645e05969db => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:55.905380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516255e9-d864-4950-9480-6a8edfd74739 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:55.905917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516255e9-d864-4950-9480-6a8edfd74739 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:55.906010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516255e9-d864-4950-9480-6a8edfd74739 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:55.910441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:23:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:55.916153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e0269-6daf-413b-91de-d5b161174b1a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:55.916567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e0269-6daf-413b-91de-d5b161174b1a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:55.916740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e0269-6daf-413b-91de-d5b161174b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:55.936151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-538e0269-6daf-413b-91de-d5b161174b1a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:55.937744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-538e0269-6daf-413b-91de-d5b161174b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:55.946873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 10:23:55.969100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516255e9-d864-4950-9480-6a8edfd74739 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:55.982664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516255e9-d864-4950-9480-6a8edfd74739 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:55.987084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42e844-18b9-46bc-8f01-b7618f1ad6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:55.987721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42e844-18b9-46bc-8f01-b7618f1ad6db admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:55.988014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42e844-18b9-46bc-8f01-b7618f1ad6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:55.988294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42e844-18b9-46bc-8f01-b7618f1ad6db admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:55.997073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42e844-18b9-46bc-8f01-b7618f1ad6db admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:55.998864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42e844-18b9-46bc-8f01-b7618f1ad6db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.008061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Jul 18 10:23:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:23:56.013728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e0269-6daf-413b-91de-d5b161174b1a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.023187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e0269-6daf-413b-91de-d5b161174b1a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.027747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b311a03-12cd-4d2e-8925-2e7115f37b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.028606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b311a03-12cd-4d2e-8925-2e7115f37b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.028927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b311a03-12cd-4d2e-8925-2e7115f37b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.029201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b311a03-12cd-4d2e-8925-2e7115f37b0a admin admin] PATH_INFO: `/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.030604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b311a03-12cd-4d2e-8925-2e7115f37b0a admin admin] RBAC: Authorizing `identity:get_user(user_id=bf6162d0fbf040d78eb20f2b6ea7ba47)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.032449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b311a03-12cd-4d2e-8925-2e7115f37b0a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.034065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:58.121966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a502f71-c578-48df-b341-df1b59975b98 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.122496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a502f71-c578-48df-b341-df1b59975b98 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.122588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a502f71-c578-48df-b341-df1b59975b98 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.123934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:58.129507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ad676-86e6-444f-b3ce-7e96c9de8e44 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.129930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ad676-86e6-444f-b3ce-7e96c9de8e44 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.130195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ad676-86e6-444f-b3ce-7e96c9de8e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.147998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c94ad676-86e6-444f-b3ce-7e96c9de8e44 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:58.149264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c94ad676-86e6-444f-b3ce-7e96c9de8e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:58.184530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38212a70-ad8a-4e28-91b7-8670b7c2b096 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.184892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38212a70-ad8a-4e28-91b7-8670b7c2b096 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.185218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38212a70-ad8a-4e28-91b7-8670b7c2b096 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.186795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:58.193608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.193965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.194286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.240461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80394) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 10:23:58.241138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:23:58.241279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80394) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 10:23:58.263763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ad676-86e6-444f-b3ce-7e96c9de8e44 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.286510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:58.287269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:58.288848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:58.370177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ad676-86e6-444f-b3ce-7e96c9de8e44 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:58.390620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:23:58.405199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:58.406003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.406285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.406560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.412755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.418885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:58.431044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:58.432031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:23:58.433213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:58.438061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.520255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6f72-61ea-4e9b-b49e-f55bfccc66c5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:58.522091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:58.523128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:58.541692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e5b21-c4ad-470a-9d29-5c3fb43ce875 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:58.558775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:58.559674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.559962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.560238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.577240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:58.577923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.578210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.578487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.579255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:58.579564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:58.581016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:58.593480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:58.598996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.632276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:58.633973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:58.700481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e131299-85e8-4ff5-a491-6cfc9ae948d7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:58.735693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:23:58.738206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:58.739090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.739378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.739711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] PATH_INFO: `/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.741066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] RBAC: Authorizing `identity:get_user(user_id=bf6162d0fbf040d78eb20f2b6ea7ba47)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:58.742177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.742585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:58.744217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:58.745630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:58.749833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.764920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:58.842859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8ddf7-b404-4172-821e-0b34eb81538f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:58.864578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558a145-73f3-4d98-97fe-0be377ec751f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:58.876549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:58.877375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.877660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.877934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.897579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:58.899068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:58.899971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:58.900823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:58.901788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:58.901788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:58.902682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:58.904148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:58.905829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 10/18] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:58.912831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:58.922757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:58.923618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:58.924403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:58.924403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:58.924403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:58.924403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:23:58.924403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 9/19] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:58.929808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:58.987309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:59.003341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:59.022505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea43fa-4e2f-4399-a7b6-3f879a5fefbf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:59.034777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6729b-69ea-4108-bca1-9d8acb36e777 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:59.057575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2e124-780a-4d6e-9996-00004ab4cacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:59.058239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2e124-780a-4d6e-9996-00004ab4cacb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.058517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2e124-780a-4d6e-9996-00004ab4cacb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.058803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2e124-780a-4d6e-9996-00004ab4cacb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.059648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2e124-780a-4d6e-9996-00004ab4cacb None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.domain.id=default, user.id=bf6162d0fbf040d78eb20f2b6ea7ba47) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:59.059960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2e124-780a-4d6e-9996-00004ab4cacb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:59.061482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2e124-780a-4d6e-9996-00004ab4cacb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:59.070037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 11/20] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/role_assignments?user.id=bf6162d0fbf040d78eb20f2b6ea7ba47&scope.domain.id=default&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:59.073201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d91df7-ff2a-4c6b-9e58-d43e28875862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:59.073883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d91df7-ff2a-4c6b-9e58-d43e28875862 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.074199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d91df7-ff2a-4c6b-9e58-d43e28875862 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.074506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d91df7-ff2a-4c6b-9e58-d43e28875862 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.075353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d91df7-ff2a-4c6b-9e58-d43e28875862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:59.075692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d91df7-ff2a-4c6b-9e58-d43e28875862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:59.077260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d91df7-ff2a-4c6b-9e58-d43e28875862 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:59.097636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 10/21] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:01.498421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7683a-73f1-4d98-8084-0221f02b30e9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.498963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7683a-73f1-4d98-8084-0221f02b30e9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.499039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7683a-73f1-4d98-8084-0221f02b30e9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.500473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 12/22] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:01.506400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468d17-32ca-4eb1-87a8-a9b835ac0505 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.506859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468d17-32ca-4eb1-87a8-a9b835ac0505 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.506943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468d17-32ca-4eb1-87a8-a9b835ac0505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.533745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8d468d17-32ca-4eb1-87a8-a9b835ac0505 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:01.535115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d468d17-32ca-4eb1-87a8-a9b835ac0505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:01.630135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 11/23] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:01.674285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7683a-73f1-4d98-8084-0221f02b30e9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.683699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ab6be-c404-4c3a-935b-3222f685ce85 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.684043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ab6be-c404-4c3a-935b-3222f685ce85 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.684339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ab6be-c404-4c3a-935b-3222f685ce85 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.686497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 12/24] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:01.693970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37145d-f8e0-4fd4-b299-eecab3c8637d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.696766 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37145d-f8e0-4fd4-b299-eecab3c8637d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.696766 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37145d-f8e0-4fd4-b299-eecab3c8637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.716178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4f37145d-f8e0-4fd4-b299-eecab3c8637d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:01.717866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4f37145d-f8e0-4fd4-b299-eecab3c8637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:01.774825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:01.808160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 13/25] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:01.836073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37145d-f8e0-4fd4-b299-eecab3c8637d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.886583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7683a-73f1-4d98-8084-0221f02b30e9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.954938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3860e-943e-4879-b610-eb31f634ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:01.955756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3860e-943e-4879-b610-eb31f634ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.956040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3860e-943e-4879-b610-eb31f634ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.956317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3860e-943e-4879-b610-eb31f634ccf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.964226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37145d-f8e0-4fd4-b299-eecab3c8637d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.979056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3860e-943e-4879-b610-eb31f634ccf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5ed3860e-943e-4879-b610-eb31f634ccf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:01.999147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:02.000238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:02.001179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 13/26] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:02.001872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd410c-aaee-49ea-8963-0b96aff532b8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.001942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd410c-aaee-49ea-8963-0b96aff532b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.002343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd410c-aaee-49ea-8963-0b96aff532b8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.004629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 14/27] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:02.009117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.010021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.010483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.010982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.012026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.013961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.014628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca53753-ec68-4813-aa89-1e547c8b4c31 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.014925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca53753-ec68-4813-aa89-1e547c8b4c31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.015212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca53753-ec68-4813-aa89-1e547c8b4c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.035621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.043073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bca53753-ec68-4813-aa89-1e547c8b4c31 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:02.044291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bca53753-ec68-4813-aa89-1e547c8b4c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:02.113666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 15/28] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:02.114610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca53753-ec68-4813-aa89-1e547c8b4c31 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.221311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca53753-ec68-4813-aa89-1e547c8b4c31 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.298883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.299700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.299992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.300270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.301061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:02.301371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.303428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.304960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.311775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 14/29] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 18 10:24:01 2022] POST /identity/v3/roles => generated 219 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:02.312793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.323021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 16/30] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:02.329430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.368837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.445872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e519ed-3bda-4395-8446-1311b2bca092 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.485163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06ba9f-2823-42ed-a80b-dabc58c8f86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.486344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06ba9f-2823-42ed-a80b-dabc58c8f86d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.486837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06ba9f-2823-42ed-a80b-dabc58c8f86d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.487317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06ba9f-2823-42ed-a80b-dabc58c8f86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.499725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06ba9f-2823-42ed-a80b-dabc58c8f86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.501833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06ba9f-2823-42ed-a80b-dabc58c8f86d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3a06ba9f-2823-42ed-a80b-dabc58c8f86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:24:02.522888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:02.524208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:02.525493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:24:02.525493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:24:02.525493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:02.525493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:02.527714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 15/31] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:02.529594 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a345dc-6606-4fb4-9260-659d67dfbe3e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.530168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a345dc-6606-4fb4-9260-659d67dfbe3e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.530651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a345dc-6606-4fb4-9260-659d67dfbe3e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.532251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 16/32] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:02.533278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a345dc-6606-4fb4-9260-659d67dfbe3e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.579336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec750d30-5196-4a78-ba87-4c130ace7adf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.623052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b04ca-1613-483f-8779-6804fbb02a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.623928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b04ca-1613-483f-8779-6804fbb02a0c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.624518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b04ca-1613-483f-8779-6804fbb02a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.624985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b04ca-1613-483f-8779-6804fbb02a0c None admin] PATH_INFO: `/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.626509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b04ca-1613-483f-8779-6804fbb02a0c None admin] RBAC: Authorizing `identity:get_user(user_id=bf6162d0fbf040d78eb20f2b6ea7ba47)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.628216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b04ca-1613-483f-8779-6804fbb02a0c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.631550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47 => generated 247 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:02.633072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4abac-719f-4201-acc7-b31f858cc3f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.633646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4abac-719f-4201-acc7-b31f858cc3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.634061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4abac-719f-4201-acc7-b31f858cc3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.637765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.652687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a345dc-6606-4fb4-9260-659d67dfbe3e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.655164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-26b4abac-719f-4201-acc7-b31f858cc3f2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:02.656696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26b4abac-719f-4201-acc7-b31f858cc3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:02.704749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.708812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.708812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.708812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.708812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:02.708812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.709160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.726710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 17/34] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:02.726710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.754548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.765006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 18/35] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:02.767175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4abac-719f-4201-acc7-b31f858cc3f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.854216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d43626-1947-44cd-9b69-62aa0ca2d694 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.893023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.893967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.894486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.894864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.896550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.898264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.898428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4abac-719f-4201-acc7-b31f858cc3f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.900267 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 18/36] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:02.905678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.927154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.944070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1fdfa-fdf1-491f-a913-5a0c0a675d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.944885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1fdfa-fdf1-491f-a913-5a0c0a675d99 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.945337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1fdfa-fdf1-491f-a913-5a0c0a675d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.945861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1fdfa-fdf1-491f-a913-5a0c0a675d99 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.947522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1fdfa-fdf1-491f-a913-5a0c0a675d99 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.949135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1fdfa-fdf1-491f-a913-5a0c0a675d99 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.019971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a091-d187-40f2-bba1-ef8e3d76c318 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:03.056728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:03.057481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.057778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.058059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.058817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:03.060813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.148656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 19/37] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 18 10:24:02 2022] POST /identity/v3/roles => generated 225 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:03.149568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:03.178372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 19/38] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:24:02 2022] POST /identity/v3/projects => generated 301 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:03.179903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7187c-e7f2-48b3-9548-8bc2bbbf9761 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.180205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7187c-e7f2-48b3-9548-8bc2bbbf9761 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.180495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7187c-e7f2-48b3-9548-8bc2bbbf9761 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.182008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 20/39] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:03.191471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792333dd-f493-4ce6-9d1e-e2f3fa45c4b6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.191812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792333dd-f493-4ce6-9d1e-e2f3fa45c4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.196106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792333dd-f493-4ce6-9d1e-e2f3fa45c4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.239117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-792333dd-f493-4ce6-9d1e-e2f3fa45c4b6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:03.240974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-792333dd-f493-4ce6-9d1e-e2f3fa45c4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:03.325055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:03.335423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:03.362824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363dbaf-e9b8-4e11-a85a-ad9db8719ea7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:03.368585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 21/40] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:03.370150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022088b-2a51-4b57-8af4-74368aee34be None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.370546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022088b-2a51-4b57-8af4-74368aee34be None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.370934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022088b-2a51-4b57-8af4-74368aee34be None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.372700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 22/41] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:03.378672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc6c6b-676a-4eb9-a517-314c03c26736 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.379369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc6c6b-676a-4eb9-a517-314c03c26736 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.380049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc6c6b-676a-4eb9-a517-314c03c26736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.403364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fdfc6c6b-676a-4eb9-a517-314c03c26736 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:03.405141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fdfc6c6b-676a-4eb9-a517-314c03c26736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:03.416076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:03.417617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.418080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.418543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] PATH_INFO: `/v3/domains/default/users/bf6162d0fbf040d78eb20f2b6ea7ba47/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.421708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bf6162d0fbf040d78eb20f2b6ea7ba47, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:03.423229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.483372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 23/42] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:03.484202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc6c6b-676a-4eb9-a517-314c03c26736 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:03.513437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 20/43] 10.222.0.49 () {58 vars in 1361 bytes} [Mon Jul 18 10:24:03 2022] PUT /identity/v3/domains/default/users/bf6162d0fbf040d78eb20f2b6ea7ba47/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:03.514680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:03.602068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc6c6b-676a-4eb9-a517-314c03c26736 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:03.636460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446098d-219d-4677-845a-a13d8f5a33cc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:03.686784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:03.687616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.689175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.689175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.689565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:03.691145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.693752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 21/44] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:03.700674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:03.705236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:03.727199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:03.756930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81479888-d591-474e-998d-3155de573f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:03.757612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81479888-d591-474e-998d-3155de573f03 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.757900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81479888-d591-474e-998d-3155de573f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.758178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81479888-d591-474e-998d-3155de573f03 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.758935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81479888-d591-474e-998d-3155de573f03 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:03.760479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81479888-d591-474e-998d-3155de573f03 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.819074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509ed8a-c681-47c2-a9a2-787c1a1999ba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:03.860898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb762b8-853f-4daa-92b9-7edcd81259b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:03.861680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb762b8-853f-4daa-92b9-7edcd81259b5 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.862072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb762b8-853f-4daa-92b9-7edcd81259b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.862516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb762b8-853f-4daa-92b9-7edcd81259b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.863401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb762b8-853f-4daa-92b9-7edcd81259b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:03.865801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb762b8-853f-4daa-92b9-7edcd81259b5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.920146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 24/45] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 18 10:24:03 2022] POST /identity/v3/roles => generated 223 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:03.972571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 22/46] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:24:03 2022] POST /identity/v3/projects => generated 312 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:24:05.606761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430fa3d-960b-40c9-b393-d0054a84b74c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.607334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430fa3d-960b-40c9-b393-d0054a84b74c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.607334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430fa3d-960b-40c9-b393-d0054a84b74c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.608713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 25/47] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:05.614025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff99490-8312-4dc2-be29-7c8c7c0de123 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.614479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff99490-8312-4dc2-be29-7c8c7c0de123 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.614589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff99490-8312-4dc2-be29-7c8c7c0de123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.636826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5ff99490-8312-4dc2-be29-7c8c7c0de123 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:05.638387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5ff99490-8312-4dc2-be29-7c8c7c0de123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:05.694151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.717170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 23/48] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:05.751047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430fa3d-960b-40c9-b393-d0054a84b74c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.862200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430fa3d-960b-40c9-b393-d0054a84b74c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.897330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59631f-c868-44fa-93b8-f765c5ba4402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.898127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59631f-c868-44fa-93b8-f765c5ba4402 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.898416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59631f-c868-44fa-93b8-f765c5ba4402 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.898695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59631f-c868-44fa-93b8-f765c5ba4402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.909364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59631f-c868-44fa-93b8-f765c5ba4402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ee59631f-c868-44fa-93b8-f765c5ba4402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:05.918161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:05.919062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:05.919890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 26/49] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:05.925346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff99490-8312-4dc2-be29-7c8c7c0de123 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.025385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff99490-8312-4dc2-be29-7c8c7c0de123 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.045318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f382b2-2f57-4d2c-9843-311b11a07ca9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.045612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f382b2-2f57-4d2c-9843-311b11a07ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.045900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f382b2-2f57-4d2c-9843-311b11a07ca9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.047281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 27/50] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:06.052932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bf84-b8ef-43cd-bab2-3efd81a94218 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.053263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bf84-b8ef-43cd-bab2-3efd81a94218 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.053500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bf84-b8ef-43cd-bab2-3efd81a94218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.061346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.062016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.062304 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.062587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.063344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:06.063653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.065069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.070304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0750bf84-b8ef-43cd-bab2-3efd81a94218 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:06.071549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0750bf84-b8ef-43cd-bab2-3efd81a94218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:06.077040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 24/51] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:06.082558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.096014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.159662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 28/52] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:06.185950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750bf84-b8ef-43cd-bab2-3efd81a94218 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.242799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.297296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750bf84-b8ef-43cd-bab2-3efd81a94218 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.299723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86d2c-ee02-4633-8541-715ab41effdc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.332982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.334433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.334861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.335036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.336185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.337703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.339027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.339318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 29/53] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:06.340431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.341043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.341555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] PATH_INFO: `/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.343638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] RBAC: Authorizing `identity:get_user(user_id=bf6162d0fbf040d78eb20f2b6ea7ba47)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.346370 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.347781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.348544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 25/54] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:06.354914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.377620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.456851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab543da-f157-47a5-b78e-367f47a7c5d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.463490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab975c-3bb6-4290-88ea-6c9d4cd2abef None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.480601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.491721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fced5b-9933-48a4-898c-2ceefd6643d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.496437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fced5b-9933-48a4-898c-2ceefd6643d6 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.496437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fced5b-9933-48a4-898c-2ceefd6643d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.496437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fced5b-9933-48a4-898c-2ceefd6643d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.496437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fced5b-9933-48a4-898c-2ceefd6643d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.496437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fced5b-9933-48a4-898c-2ceefd6643d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.500555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.501433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.501793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.502110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.503417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.505324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.507970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 26/55] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:06.514059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.617968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f533d-0ebe-44d7-a619-86ed2e2af405 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.635516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 30/56] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Jul 18 10:24:06 2022] POST /identity/v3/projects => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:24:06.659190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4f631-cb22-4f1b-b6ec-003a75979210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.660001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4f631-cb22-4f1b-b6ec-003a75979210 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.660356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4f631-cb22-4f1b-b6ec-003a75979210 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.660660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4f631-cb22-4f1b-b6ec-003a75979210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.661741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4f631-cb22-4f1b-b6ec-003a75979210 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.domain.id=default, user.id=bf6162d0fbf040d78eb20f2b6ea7ba47) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:06.662298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4f631-cb22-4f1b-b6ec-003a75979210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.664211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4f631-cb22-4f1b-b6ec-003a75979210 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.675328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 27/57] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/role_assignments?user.id=bf6162d0fbf040d78eb20f2b6ea7ba47&scope.domain.id=default&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:08.701881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cb674-9c52-4d9f-9638-61732e85fb1d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:08.702498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cb674-9c52-4d9f-9638-61732e85fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:08.702592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cb674-9c52-4d9f-9638-61732e85fb1d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:08.704106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 31/58] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:08.709295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d627-45db-49fb-9fe9-6e7e9b8e4d28 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:08.709736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d627-45db-49fb-9fe9-6e7e9b8e4d28 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:08.709860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d627-45db-49fb-9fe9-6e7e9b8e4d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:08.727420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-32e3d627-45db-49fb-9fe9-6e7e9b8e4d28 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:08.728660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-32e3d627-45db-49fb-9fe9-6e7e9b8e4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:08.773513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:08.796513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 28/59] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:08.821860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cb674-9c52-4d9f-9638-61732e85fb1d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:08.903127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:08.928163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cb674-9c52-4d9f-9638-61732e85fb1d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:08.966201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf249e4-2a41-4379-b079-ebf32b094314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:08.966925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf249e4-2a41-4379-b079-ebf32b094314 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:08.967216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf249e4-2a41-4379-b079-ebf32b094314 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:08.967499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf249e4-2a41-4379-b079-ebf32b094314 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:08.968685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf249e4-2a41-4379-b079-ebf32b094314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:08.970205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf249e4-2a41-4379-b079-ebf32b094314 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:08.971951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 32/60] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:08.977871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3d627-45db-49fb-9fe9-6e7e9b8e4d28 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:09.079266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3d627-45db-49fb-9fe9-6e7e9b8e4d28 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:09.113738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1e312-14f9-4eeb-9144-f21c9410e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:09.114416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1e312-14f9-4eeb-9144-f21c9410e8e0 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:09.114695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1e312-14f9-4eeb-9144-f21c9410e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:09.116015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1e312-14f9-4eeb-9144-f21c9410e8e0 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:09.116736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1e312-14f9-4eeb-9144-f21c9410e8e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:09.118329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1e312-14f9-4eeb-9144-f21c9410e8e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:09.215286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 29/61] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:08 2022] POST /identity/v3/users => generated 275 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:12.503515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba20d-f215-47b0-a3de-ad47d5227882 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.504754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba20d-f215-47b0-a3de-ad47d5227882 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.505308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba20d-f215-47b0-a3de-ad47d5227882 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.508120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 33/62] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:12.513474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917f581-85ba-4d74-a937-93901c635e7b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.513903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917f581-85ba-4d74-a937-93901c635e7b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.514034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917f581-85ba-4d74-a937-93901c635e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.538484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6917f581-85ba-4d74-a937-93901c635e7b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:12.540045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6917f581-85ba-4d74-a937-93901c635e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:12.546833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2243c-14bc-4783-954b-6fc77d49ffec None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.547466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2243c-14bc-4783-954b-6fc77d49ffec None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.547953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2243c-14bc-4783-954b-6fc77d49ffec None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.549921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 34/63] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:12.556074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fed4c8-49d6-43d0-b1bb-5e43973fa879 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.556674 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fed4c8-49d6-43d0-b1bb-5e43973fa879 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.557188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fed4c8-49d6-43d0-b1bb-5e43973fa879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.586429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-66fed4c8-49d6-43d0-b1bb-5e43973fa879 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:12.587635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66fed4c8-49d6-43d0-b1bb-5e43973fa879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:12.592715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:12.656139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 30/64] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:12.657409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524ff46-84be-4053-8c85-37aee65bca8b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.657702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524ff46-84be-4053-8c85-37aee65bca8b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.662667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524ff46-84be-4053-8c85-37aee65bca8b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.662667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 31/65] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:12.668988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4f70-2187-4975-a1c8-7763d99e9dff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.669534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4f70-2187-4975-a1c8-7763d99e9dff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.670027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4f70-2187-4975-a1c8-7763d99e9dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.691508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-49df4f70-2187-4975-a1c8-7763d99e9dff None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:12.694092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-49df4f70-2187-4975-a1c8-7763d99e9dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:12.704072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 35/66] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:12.710302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fed4c8-49d6-43d0-b1bb-5e43973fa879 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:12.793093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 32/67] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:12.796579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df4f70-2187-4975-a1c8-7763d99e9dff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:12.834983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:12.862311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fed4c8-49d6-43d0-b1bb-5e43973fa879 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:12.908100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:12.909087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.909380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.909766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.921716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:12.930946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:12.931877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:12.932665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:12.933294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:12.936990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 36/68] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:12.938649 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:12.951586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df4f70-2187-4975-a1c8-7763d99e9dff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:12.979558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562857a5-cfe5-4f77-8d10-b535ac45a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:12.980404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562857a5-cfe5-4f77-8d10-b535ac45a679 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.980715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562857a5-cfe5-4f77-8d10-b535ac45a679 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.981061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562857a5-cfe5-4f77-8d10-b535ac45a679 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.999259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562857a5-cfe5-4f77-8d10-b535ac45a679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-562857a5-cfe5-4f77-8d10-b535ac45a679 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:13.010290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:13.011309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:13.012106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 33/69] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:13.012837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f9710-5e5a-4d69-a970-2835164b8467 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.013131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f9710-5e5a-4d69-a970-2835164b8467 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.013418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f9710-5e5a-4d69-a970-2835164b8467 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.015034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 34/70] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:13.019497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f9710-5e5a-4d69-a970-2835164b8467 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.070966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da2fb7-8dc4-4ffa-88bc-e954f8f0ef4b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.108368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.109489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.109985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.110230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.112270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.123675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:13.132663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:13.133588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:13.134366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 37/71] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:13.135015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.141848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f9710-5e5a-4d69-a970-2835164b8467 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.178238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bff1d-29ed-442c-bbd7-b99c3affd48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.179140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bff1d-29ed-442c-bbd7-b99c3affd48c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.179470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bff1d-29ed-442c-bbd7-b99c3affd48c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.179752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bff1d-29ed-442c-bbd7-b99c3affd48c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.180616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bff1d-29ed-442c-bbd7-b99c3affd48c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:13.180954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bff1d-29ed-442c-bbd7-b99c3affd48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:13.182371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bff1d-29ed-442c-bbd7-b99c3affd48c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:13.191083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 35/72] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:13.192290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fdfdc-73a2-448d-814b-bacf6ce1ef31 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.192581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fdfdc-73a2-448d-814b-bacf6ce1ef31 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.192885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fdfdc-73a2-448d-814b-bacf6ce1ef31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.211269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c48fdfdc-73a2-448d-814b-bacf6ce1ef31 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:13.212538 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c48fdfdc-73a2-448d-814b-bacf6ce1ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:13.223896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.243791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edcc4b-1327-46e3-a4c0-eefde13e1b12 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.278526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f3c50-f7fc-460e-aaad-47dd7275b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.279206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3c50-f7fc-460e-aaad-47dd7275b960 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.279489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3c50-f7fc-460e-aaad-47dd7275b960 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.279770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3c50-f7fc-460e-aaad-47dd7275b960 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.280526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3c50-f7fc-460e-aaad-47dd7275b960 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:13.280863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3c50-f7fc-460e-aaad-47dd7275b960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:13.288826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 36/73] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:13.288990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3c50-f7fc-460e-aaad-47dd7275b960 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:13.289111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fdfdc-73a2-448d-814b-bacf6ce1ef31 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.303616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 38/74] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:13.303616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2603bb-b5f3-451a-9f50-ed72f6838b38 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.303616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2603bb-b5f3-451a-9f50-ed72f6838b38 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.303616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2603bb-b5f3-451a-9f50-ed72f6838b38 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.303616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 39/75] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:13.305561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2603bb-b5f3-451a-9f50-ed72f6838b38 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.363363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.389237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fdfdc-73a2-448d-814b-bacf6ce1ef31 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.409122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2603bb-b5f3-451a-9f50-ed72f6838b38 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.424668 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.425369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.425656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.425937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.426754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:13.427068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:13.429033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:13.437067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 37/76] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:13.437888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.446486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.447371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.447696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.447963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] PATH_INFO: `/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.449306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] RBAC: Authorizing `identity:get_user(user_id=bf6162d0fbf040d78eb20f2b6ea7ba47)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:13.450735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:13.451890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.452375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 40/77] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:13.453173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.546466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517051d-0136-4eba-b78a-0d4316588e26 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.567207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd850ca-0611-479b-8812-494a88f5bb54 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.582167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba918c-aad9-4a0e-91cd-32833d5501ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.583050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba918c-aad9-4a0e-91cd-32833d5501ef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.583338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba918c-aad9-4a0e-91cd-32833d5501ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.584165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba918c-aad9-4a0e-91cd-32833d5501ef None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.596550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.600573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba918c-aad9-4a0e-91cd-32833d5501ef None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:13.602167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba918c-aad9-4a0e-91cd-32833d5501ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:13.603950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 38/78] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:13.610342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbcefb-3985-461a-bc28-a0f5013c1415 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.610342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbcefb-3985-461a-bc28-a0f5013c1415 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.610342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbcefb-3985-461a-bc28-a0f5013c1415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.610575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.610575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.610888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.611171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.623043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7cdbcefb-3985-461a-bc28-a0f5013c1415 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:13.624181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7cdbcefb-3985-461a-bc28-a0f5013c1415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:13.624942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:13.634171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:13.635239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:13.636143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 41/79] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:13.636806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.687998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.696489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 39/80] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:13.697485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbcefb-3985-461a-bc28-a0f5013c1415 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.744695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de218e-9eba-4e95-942d-67b023e50992 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.782835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.783760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.784049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.784333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.785632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:13.787489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:13.793788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 42/81] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:13.793788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:13.807890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbcefb-3985-461a-bc28-a0f5013c1415 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.844274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.844972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.845267 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.845558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:13.845695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.894327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08eacc-a41a-420f-b62e-494806905f39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:13.910791 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:13.918471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:13.920015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:13.933333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:13.933995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:13.934279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:13.934613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.007051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.008560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.031452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 40/82] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:14.032337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.061350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.111052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 43/83] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:14.112159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.136723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.142435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5a39b-2836-4e15-87c0-af2cfc6da91b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.177963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.178642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.178933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.179212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.179967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:14.180277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.181718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.189939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 41/84] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.190752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.221205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0219-5592-436c-ac89-7fd338e199c8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.260804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.261626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.261913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.262190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.263375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.264899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.266529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 44/85] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.267298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.292216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.298272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94a80d-bb86-4880-b23d-481d1a259dba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.332876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.334043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.334368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.334651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.362617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.373339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade24679-7f11-4dab-acd9-712bbe8e432c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.411243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.413474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.413587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.414537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.414790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.415068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.416290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.417779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.419495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 45/86] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.420275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.509979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.517581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 42/87] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:14.518491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.525238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19feb7-4d66-4160-869c-afddd1d0e4b7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.562725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.563584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.563871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.564146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.565654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.567148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.569062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 46/88] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.569845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.586367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.628837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d362-a677-4aac-8d68-4ad18380ee7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.670861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.671687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.671921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.672202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.673484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.674993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.676729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 43/89] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.677605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.678732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a87960-322d-410f-973c-75b6d61cebc9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.717733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.718399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.718683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.718959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.719687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.721187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.781910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce12a68-13c0-4ecf-a0ee-5501c6a88488 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.811742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.816277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 47/90] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:14 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:14.817114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.823118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.825090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.825090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.825090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.825361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=bf6162d0fbf040d78eb20f2b6ea7ba47) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:14.825625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.827870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.836789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.841536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 44/91] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/role_assignments?user.id=bf6162d0fbf040d78eb20f2b6ea7ba47&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.842245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.986930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2273acc-2109-4144-a44c-e4ebdc3d119f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.989561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8e5a7-612e-4e50-a8db-3c1bfc6a33c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:15.041642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:15.041947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:15.042464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:15.042575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:15.042759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:15.042864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:15.043045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:15.043143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:15.043971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=d024a945bdaf4746aa6cddd29acc851e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:15.044293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:15.044387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:15.045819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:15.045937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:15.047607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 48/92] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:15.048397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:15.059802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 45/93] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/role_assignments?user.id=d024a945bdaf4746aa6cddd29acc851e&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:15.060620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:15.155205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:15.179804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:15.213050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764c2fa-4205-44be-928c-c7d413a8dfba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:15.224451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527601bc-199d-4203-92a5-87293a236f03 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:15.277088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:15.278282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:15.278885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:15.279312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:15.280157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:15.280618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] RBAC: Adding query filter params (role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=d024a945bdaf4746aa6cddd29acc851e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:15.281113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:15.281224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:15.281599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:15.281978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:15.283434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:15.293941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 46/94] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:15 2022] GET /identity/v3/role_assignments?user.id=d024a945bdaf4746aa6cddd29acc851e&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=4f582d9851794ff894a232e045525ec4 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:15.383358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:15.385100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:15.492432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:15.515581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 49/95] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:15.521200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:15.687654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3064898-cefd-4ca5-bade-41de9dd15a7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:15.753477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:15.775463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f47f16-cb25-437e-ad7c-b8d1c00094f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:15.776348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47f16-cb25-437e-ad7c-b8d1c00094f0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:15.776632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47f16-cb25-437e-ad7c-b8d1c00094f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:15.776939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47f16-cb25-437e-ad7c-b8d1c00094f0 None admin] PATH_INFO: `/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:15.778163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f47f16-cb25-437e-ad7c-b8d1c00094f0 None admin] RBAC: Authorizing `identity:get_project(project_id=1a5b6641f18e4b10ac87673e3eee0ebb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:15.779656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f47f16-cb25-437e-ad7c-b8d1c00094f0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:15.781680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 47/96] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:15 2022] GET /identity/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:15.789916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:15.941483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f8cad-9a88-4404-9570-27c15302a802 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:15.988295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0631e7b-aad9-4eb1-a791-bbec12d49690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:15.989024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0631e7b-aad9-4eb1-a791-bbec12d49690 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:15.989353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0631e7b-aad9-4eb1-a791-bbec12d49690 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:15.989637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0631e7b-aad9-4eb1-a791-bbec12d49690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:15.990786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0631e7b-aad9-4eb1-a791-bbec12d49690 None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=1a5b6641f18e4b10ac87673e3eee0ebb, user.id=d024a945bdaf4746aa6cddd29acc851e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:15.991105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0631e7b-aad9-4eb1-a791-bbec12d49690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:15.992588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0631e7b-aad9-4eb1-a791-bbec12d49690 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:16.001339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 50/97] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:15 2022] GET /identity/v3/role_assignments?user.id=d024a945bdaf4746aa6cddd29acc851e&scope.project.id=1a5b6641f18e4b10ac87673e3eee0ebb&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:17.578548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6752aa-bdac-4947-a427-004ae6c6d7cb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.579056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6752aa-bdac-4947-a427-004ae6c6d7cb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.579158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6752aa-bdac-4947-a427-004ae6c6d7cb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.580539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 48/98] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:17.586858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f3e8f-e599-4e2d-a968-e45a308b9050 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.587310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f3e8f-e599-4e2d-a968-e45a308b9050 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.587514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f3e8f-e599-4e2d-a968-e45a308b9050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.604927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6d7f3e8f-e599-4e2d-a968-e45a308b9050 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:17.606142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6d7f3e8f-e599-4e2d-a968-e45a308b9050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:17.660157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:17.695012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 51/99] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:17.729591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6752aa-bdac-4947-a427-004ae6c6d7cb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:17.932463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6752aa-bdac-4947-a427-004ae6c6d7cb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:17.958431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c8456-36ac-41a0-a5d4-47ae7aa8f6fc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.959090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c8456-36ac-41a0-a5d4-47ae7aa8f6fc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.959598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c8456-36ac-41a0-a5d4-47ae7aa8f6fc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.961235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 52/100] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:17.966792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a074a-cdc8-4f6d-8851-2aea3c348ca6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.967362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a074a-cdc8-4f6d-8851-2aea3c348ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.967849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a074a-cdc8-4f6d-8851-2aea3c348ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.973222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:17.974129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.975017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.975302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.986554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:17.996269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:17.997260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:17.998038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:17.999532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 49/101] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:18.004038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:18.011566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-972a074a-cdc8-4f6d-8851-2aea3c348ca6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:18.013944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-972a074a-cdc8-4f6d-8851-2aea3c348ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:18.113501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 53/102] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:18.114734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e147a-c776-4796-a635-24039879c355 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.115571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e147a-c776-4796-a635-24039879c355 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.116269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e147a-c776-4796-a635-24039879c355 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.117750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 54/103] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:18.123404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817be228-4241-4f49-aa4d-7db046325d73 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.123699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817be228-4241-4f49-aa4d-7db046325d73 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.123978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817be228-4241-4f49-aa4d-7db046325d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.138065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:18.143051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-817be228-4241-4f49-aa4d-7db046325d73 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:18.144307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-817be228-4241-4f49-aa4d-7db046325d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:18.232301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 55/104] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:18.236924 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817be228-4241-4f49-aa4d-7db046325d73 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:18.269652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564eef0-5219-4cfd-ac4c-fc487ef19ba9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:18.323914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:18.324586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.324902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.325243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.325522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:18.326289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:18.326394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:18.329201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:18.346100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 50/105] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:18.347113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:18.469153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817be228-4241-4f49-aa4d-7db046325d73 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:18.525492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a4bc7-7e63-4679-a989-d0849d636f0f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:18.533365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:18.534761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.535194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.535613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.535994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:18.549997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:18.562109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:18.563187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:18.564129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:18.566625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 56/106] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:18.567761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:18.568221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a164c21-36e0-4a6a-b007-e4ce5a0a3682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:18.569414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a164c21-36e0-4a6a-b007-e4ce5a0a3682 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.569802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a164c21-36e0-4a6a-b007-e4ce5a0a3682 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.570179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a164c21-36e0-4a6a-b007-e4ce5a0a3682 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.572147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a164c21-36e0-4a6a-b007-e4ce5a0a3682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:18.574883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a164c21-36e0-4a6a-b007-e4ce5a0a3682 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:18.576673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 51/107] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:18.582251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f429ce3-5168-4f4d-bf96-8f09ed56c4cb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.582805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f429ce3-5168-4f4d-bf96-8f09ed56c4cb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.582805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f429ce3-5168-4f4d-bf96-8f09ed56c4cb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.584367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 52/108] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:18.585727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f429ce3-5168-4f4d-bf96-8f09ed56c4cb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:18.687516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:18.696726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d00d7-8df9-4c17-ae6a-078a0cebc6a4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:18.702403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f429ce3-5168-4f4d-bf96-8f09ed56c4cb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:18.751950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17332b6-c50a-4b3c-bdd7-b4b20a72ec5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:18.752627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17332b6-c50a-4b3c-bdd7-b4b20a72ec5b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.752949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17332b6-c50a-4b3c-bdd7-b4b20a72ec5b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.753232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17332b6-c50a-4b3c-bdd7-b4b20a72ec5b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.753995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17332b6-c50a-4b3c-bdd7-b4b20a72ec5b None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:18.754567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17332b6-c50a-4b3c-bdd7-b4b20a72ec5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:18.755642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:18.756045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17332b6-c50a-4b3c-bdd7-b4b20a72ec5b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:18.756526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.756837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.757120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] PATH_INFO: `/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.758550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] RBAC: Authorizing `identity:get_user(user_id=bf6162d0fbf040d78eb20f2b6ea7ba47)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:18.760058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:18.761897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 57/109] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:18 2022] GET /identity/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:18.762558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:18.769673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 53/110] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:18.772822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36115ca-b4f3-479a-96f3-b3a96d8f9c9e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.773375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36115ca-b4f3-479a-96f3-b3a96d8f9c9e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.773860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36115ca-b4f3-479a-96f3-b3a96d8f9c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.794539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a36115ca-b4f3-479a-96f3-b3a96d8f9c9e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:18.796062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a36115ca-b4f3-479a-96f3-b3a96d8f9c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:18.829014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:18.868362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 54/111] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:18.869692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319678-25df-44de-bfb3-55fd46b88a47 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.869991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319678-25df-44de-bfb3-55fd46b88a47 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.870278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319678-25df-44de-bfb3-55fd46b88a47 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.871684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 55/112] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:18.872440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25319678-25df-44de-bfb3-55fd46b88a47 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:18.889105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45572d86-3d11-4137-936a-ee8b159ef890 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:18.936192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d8b79-01a4-468f-8a59-3b9f5660b047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:18.936893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d8b79-01a4-468f-8a59-3b9f5660b047 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:18.937182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d8b79-01a4-468f-8a59-3b9f5660b047 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:18.937473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d8b79-01a4-468f-8a59-3b9f5660b047 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:18.938193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d8b79-01a4-468f-8a59-3b9f5660b047 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:18.939675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d8b79-01a4-468f-8a59-3b9f5660b047 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:18.990003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25319678-25df-44de-bfb3-55fd46b88a47 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.031773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.032647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.032967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.033249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.034457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:19.035920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:19.038484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 56/113] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:18 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:19.039267 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.040919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 58/114] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:24:18 2022] POST /identity/v3/projects => generated 302 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:19.042487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f906729-ddc3-48dd-88c1-e4911f5efce1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.043685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f906729-ddc3-48dd-88c1-e4911f5efce1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.050543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f906729-ddc3-48dd-88c1-e4911f5efce1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.050543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 59/115] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:19.050543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcad5c-4bd9-488d-8926-0bf9ea491b08 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.050543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcad5c-4bd9-488d-8926-0bf9ea491b08 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.050543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcad5c-4bd9-488d-8926-0bf9ea491b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.068540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f1bcad5c-4bd9-488d-8926-0bf9ea491b08 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:19.069872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1bcad5c-4bd9-488d-8926-0bf9ea491b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:19.082108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:19.086522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:19.154389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 60/116] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:19.155644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcad5c-4bd9-488d-8926-0bf9ea491b08 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.184248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845453c3-6b1f-40c1-9f41-6eb685141c96 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.229441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.230337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.230624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.230901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.237355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:19.237355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:19.237355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 57/117] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:19 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:19.237355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.279890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcad5c-4bd9-488d-8926-0bf9ea491b08 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.316702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb4ce-d4ff-4ac5-8370-c682536f0157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.317459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb4ce-d4ff-4ac5-8370-c682536f0157 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.317749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb4ce-d4ff-4ac5-8370-c682536f0157 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.318072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb4ce-d4ff-4ac5-8370-c682536f0157 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.330076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafb4ce-d4ff-4ac5-8370-c682536f0157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0bafb4ce-d4ff-4ac5-8370-c682536f0157 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:19.338036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:19.339042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:19.340108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 61/118] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:19.345270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612df35-6d75-4bcf-a124-401f8d52b203 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.345612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612df35-6d75-4bcf-a124-401f8d52b203 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.350252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612df35-6d75-4bcf-a124-401f8d52b203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.352660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:19.357300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715823c5-3cca-476f-9e36-2fcae57d7e58 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.360288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:19.364261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2612df35-6d75-4bcf-a124-401f8d52b203 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:19.366262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2612df35-6d75-4bcf-a124-401f8d52b203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:19.397051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.398056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.398454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.398737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/bf6162d0fbf040d78eb20f2b6ea7ba47/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.401043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] RBAC: Authorizing `identity:create_grant(project_id=8efd660d626b4141b5c9be1824e7e416, user_id=bf6162d0fbf040d78eb20f2b6ea7ba47, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:19.402507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:19.439327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 62/119] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:19.448964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612df35-6d75-4bcf-a124-401f8d52b203 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.507455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 58/120] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:19 2022] PUT /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/bf6162d0fbf040d78eb20f2b6ea7ba47/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:19.508756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.565866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612df35-6d75-4bcf-a124-401f8d52b203 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.613771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.616131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.616590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.616971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.619912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:19.634500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:19.643527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:19.644756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:19.645728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:19.647944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 63/121] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:19.652829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.656004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001e391-6905-482f-97b4-081f9601c18d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.706239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.707126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.707476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.707744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.709556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:19.711282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:19.713289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 59/122] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:19 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:19.713905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.759198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:19.779082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9efc-29fe-4f9e-88e6-36922af72f10 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.830915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.831606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.831897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.832181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.832994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:19.833312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:19.837337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:19.847916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12947dd-4892-4cec-8c4e-1bef13d6b945 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:19.849022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 64/123] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:19.850417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.891770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:19.892992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:19.893541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:19.894015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:19.906124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:19.915231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:19.916144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:19.917032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:19.919398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 60/124] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:19.920426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:19.922630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:19.983570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001bfac-e5bc-414f-998c-a196849fa150 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.028362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.031568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.032619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.032964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.033251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.034383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:20.034706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.036663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.040095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12066be3-362a-414b-94d1-ec1a41db10d4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.048831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 65/125] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.048831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.082281 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.083191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.083969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.084271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/d024a945bdaf4746aa6cddd29acc851e/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.086698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] RBAC: Authorizing `identity:create_grant(project_id=8efd660d626b4141b5c9be1824e7e416, user_id=d024a945bdaf4746aa6cddd29acc851e, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.088702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.169931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3e24a-1ac2-4329-8484-6e94d63cba28 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.194465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.205798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 61/126] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:19 2022] PUT /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/d024a945bdaf4746aa6cddd29acc851e/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:20.213885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.215491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.216438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.216730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.217040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] PATH_INFO: `/v3/users/55302a4556e140cfa7314d27f1f40743` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.234766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] RBAC: Authorizing `identity:get_user(user_id=55302a4556e140cfa7314d27f1f40743)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.237004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.238860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 66/127] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/users/55302a4556e140cfa7314d27f1f40743 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.240530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.399047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96415-b07c-4c5f-aa55-b3a99377661d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.427036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacddb2-f5ea-414d-990b-c94ad4663c5e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.465264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.466258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.466543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.467029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.468366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.469982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.471620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 67/128] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.472596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.477432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.478129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.478474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.478703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.479487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:20.479804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.481305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.492915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 62/129] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.493885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.512482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.527020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.589466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d04c5e-9cff-432c-8ee0-50231bb7ce82 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.604646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824735a5-c272-45a0-9b53-88424ee82e56 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.627335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.628631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.628945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.629481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.642829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.644037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.644338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.644623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.645907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.648130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.649901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 63/130] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.650702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.716461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.718153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.745238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.765293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bd336-feb6-4e6b-bbca-8126a34cf902 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.811887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.815749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.816869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.817306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.817733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.819983 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.821674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.823486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 64/131] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.824368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.949541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf6b44-71be-4624-a729-e94e4584a875 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.956700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 68/132] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:20.957730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:21.029861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:21.030689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.030979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.031256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/d024a945bdaf4746aa6cddd29acc851e/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.037447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] RBAC: Authorizing `identity:create_grant(project_id=8efd660d626b4141b5c9be1824e7e416, user_id=d024a945bdaf4746aa6cddd29acc851e, role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:21.038937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:21.113982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35b6f4-0fb7-4a24-b17a-96ae202fd0da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:21.154547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:21.155446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.155769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.156082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] PATH_INFO: `/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.157458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] RBAC: Authorizing `identity:get_project(project_id=1a5b6641f18e4b10ac87673e3eee0ebb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:21.159593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:21.161891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 69/133] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:21.162805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:21.183801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 65/134] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:20 2022] PUT /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/d024a945bdaf4746aa6cddd29acc851e/roles/4f582d9851794ff894a232e045525ec4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:21.185208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:21.185398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:21.222387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:21.286025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71104f2a-2a77-4ee7-9e3f-e39a7ef4f6d6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:21.325462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:21.326365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.326652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.326966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.328838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:21.329852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:21.331668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 70/135] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:21 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:21.338640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:21.435309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732024c6-ce6b-4ddb-91ff-fdb0d677a5de None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:21.436361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:21.458018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ac027-0fa2-4808-80f7-bb4b5d13543e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:21.478017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:21.479269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.479767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.480271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] PATH_INFO: `/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb/users/d024a945bdaf4746aa6cddd29acc851e/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.482597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a5b6641f18e4b10ac87673e3eee0ebb, user_id=d024a945bdaf4746aa6cddd29acc851e, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:21.484324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:21.496490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afdaad-8768-44f5-9928-b1d09afb23c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:21.497275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afdaad-8768-44f5-9928-b1d09afb23c1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.497563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afdaad-8768-44f5-9928-b1d09afb23c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.497917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afdaad-8768-44f5-9928-b1d09afb23c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.498739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afdaad-8768-44f5-9928-b1d09afb23c1 None admin] RBAC: Adding query filter params (role.id=21ae2370a18d4c9ebbd8a56f38d6b550, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=55302a4556e140cfa7314d27f1f40743) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:21.499100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afdaad-8768-44f5-9928-b1d09afb23c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:21.500679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afdaad-8768-44f5-9928-b1d09afb23c1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:21.508034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 71/136] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:21 2022] GET /identity/v3/role_assignments?user.id=55302a4556e140cfa7314d27f1f40743&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=21ae2370a18d4c9ebbd8a56f38d6b550 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:21.513418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b326-8610-480c-b24e-31a99a491e66 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.513714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b326-8610-480c-b24e-31a99a491e66 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.513996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b326-8610-480c-b24e-31a99a491e66 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.515773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 72/137] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:21.522242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de6ee1-e490-4344-9f29-b3c55616998f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.522532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de6ee1-e490-4344-9f29-b3c55616998f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.522996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de6ee1-e490-4344-9f29-b3c55616998f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.547724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7de6ee1-e490-4344-9f29-b3c55616998f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:21.548964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7de6ee1-e490-4344-9f29-b3c55616998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:21.575985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 66/138] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:21 2022] PUT /identity/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb/users/d024a945bdaf4746aa6cddd29acc851e/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:21.633481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 73/139] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:21.660038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:21.748440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:21.783272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fed231-acee-4905-9524-cf74d2808543 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:21.834044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb1c4f-5703-456d-aad3-e590b65a2154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:21.835166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb1c4f-5703-456d-aad3-e590b65a2154 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:21.835465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb1c4f-5703-456d-aad3-e590b65a2154 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:21.835810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb1c4f-5703-456d-aad3-e590b65a2154 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:21.837154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb1c4f-5703-456d-aad3-e590b65a2154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:21.838795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb1c4f-5703-456d-aad3-e590b65a2154 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:21.840597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 67/140] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:21.846381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de6ee1-e490-4344-9f29-b3c55616998f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:21.937237 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:21.966612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de6ee1-e490-4344-9f29-b3c55616998f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:22.005831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:22.006519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.006806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.007086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.007880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:22.009465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:22.027294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928b17-75dc-443f-91c6-72a344ee2787 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.027589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928b17-75dc-443f-91c6-72a344ee2787 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.027897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928b17-75dc-443f-91c6-72a344ee2787 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.084586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 68/141] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:22.084586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a69e0-46d9-402a-a5d1-a50e01a6c51b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.084586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a69e0-46d9-402a-a5d1-a50e01a6c51b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.084586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a69e0-46d9-402a-a5d1-a50e01a6c51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.084586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e5a69e0-46d9-402a-a5d1-a50e01a6c51b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:22.084586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e5a69e0-46d9-402a-a5d1-a50e01a6c51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:22.142454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 69/142] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:22.183361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a69e0-46d9-402a-a5d1-a50e01a6c51b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:22.204582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 74/143] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:21 2022] POST /identity/v3/users => generated 283 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:22.287865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:22.331057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a69e0-46d9-402a-a5d1-a50e01a6c51b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:22.371062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:22.371842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.372130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.372436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.386720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:22.395738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:22.397052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:22.398422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 70/144] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:22.403342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:22.559558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526c55c-c47a-4875-b7ad-35d2b4818a2e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:22.601836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:22.608317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:22.609176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.609660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.610002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:22.611127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:22.612524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:22.625392 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 75/145] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:22.631466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:22.760385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6feaa7-4c97-4faa-b170-032d3a58e920 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:22.816340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:22.829593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:22.830649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.831007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.831381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] PATH_INFO: `/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.835681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] RBAC: Authorizing `identity:get_user(user_id=bf6162d0fbf040d78eb20f2b6ea7ba47)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:22.835681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:22.836484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 71/146] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:22 2022] GET /identity/v3/users/bf6162d0fbf040d78eb20f2b6ea7ba47 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:22.842517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:23.015535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7983ecd-a421-44ef-959f-995710f10cea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:23.073373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:23.074047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:23.074383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:23.074665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:23.153018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:23.164338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:23.165856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:23.295447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 76/147] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:23.301250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:23.424219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0786a-6459-401b-a9d7-4d4b4caf0d2b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:23.475725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71612e-7a71-4eb1-badb-d77c4b7d3dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:23.476613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71612e-7a71-4eb1-badb-d77c4b7d3dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:23.476925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71612e-7a71-4eb1-badb-d77c4b7d3dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:23.477209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71612e-7a71-4eb1-badb-d77c4b7d3dc4 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:23.481568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71612e-7a71-4eb1-badb-d77c4b7d3dc4 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:23.483275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71612e-7a71-4eb1-badb-d77c4b7d3dc4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:23.485336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 72/148] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:23 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:23.492587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:23.636815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:23.638438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e356c-e6aa-4f64-9b52-ba8138fe13fe None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:23.638805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e356c-e6aa-4f64-9b52-ba8138fe13fe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:23.639090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e356c-e6aa-4f64-9b52-ba8138fe13fe None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:23.640653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 73/149] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:23.646494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a70e-5fd3-413c-b853-b65b7869e238 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:23.646785 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a70e-5fd3-413c-b853-b65b7869e238 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:23.655928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a70e-5fd3-413c-b853-b65b7869e238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:23.659388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73273c-ff9e-4ffc-b10f-0238af3ab124 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:23.681257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7713a70e-5fd3-413c-b853-b65b7869e238 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:23.682653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7713a70e-5fd3-413c-b853-b65b7869e238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:23.718694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:23.719391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:23.719680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:23.719955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:23.720851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=bf6162d0fbf040d78eb20f2b6ea7ba47) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:23.721167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:23.722663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:23.734204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 77/150] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:23 2022] GET /identity/v3/role_assignments?user.id=bf6162d0fbf040d78eb20f2b6ea7ba47&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:23.808441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:23.821082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 74/151] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:23.867023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:24.011665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652c6a0-b9a9-4d7e-8de4-ae9fb76b5f60 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:24.052154 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:24.053518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:24.053518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:24.053831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:24.074063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:24.082142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:24.083057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:24.084338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:24.085673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 78/152] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:24.090114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713a70e-5fd3-413c-b853-b65b7869e238 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:24.201361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713a70e-5fd3-413c-b853-b65b7869e238 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:24.237648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:24.238393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:24.238626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:24.238899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:24.239669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:24.240106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:24.241762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:24.250997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 75/153] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:24.257436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:24.277686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:24.383660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9f5c5-d327-4b6b-b8a9-b255ef27684d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:24.423658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc4eb0-23ca-4a1a-bb1c-5ea164e7e0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:24.424544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc4eb0-23ca-4a1a-bb1c-5ea164e7e0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:24.424845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc4eb0-23ca-4a1a-bb1c-5ea164e7e0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:24.425128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc4eb0-23ca-4a1a-bb1c-5ea164e7e0e0 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:24.427182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc4eb0-23ca-4a1a-bb1c-5ea164e7e0e0 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:24.428863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc4eb0-23ca-4a1a-bb1c-5ea164e7e0e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:24.430525 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 79/154] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:24 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:24.436264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:24.513794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:24.556657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566fab-1cc1-4add-91c0-e455519402ea None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:24.593221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:24.593883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:24.594169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:24.594449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:24.668042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:24.669564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:24.751434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:24.780399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88233352-9be2-4e87-8df9-7826166d1f8f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:24.781119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88233352-9be2-4e87-8df9-7826166d1f8f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:24.781600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88233352-9be2-4e87-8df9-7826166d1f8f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:24.783195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 80/155] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:24.789313 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3c5a8-7c8c-4187-88a9-9b1151eb7329 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:24.789896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3c5a8-7c8c-4187-88a9-9b1151eb7329 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:24.790403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3c5a8-7c8c-4187-88a9-9b1151eb7329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:24.815467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 76/156] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:24.821547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:24.823792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-91f3c5a8-7c8c-4187-88a9-9b1151eb7329 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:24.825029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-91f3c5a8-7c8c-4187-88a9-9b1151eb7329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:24.949419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520ac9e-c6c8-46d5-a768-bd843557d939 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:24.958749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 81/157] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:24.999127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3c5a8-7c8c-4187-88a9-9b1151eb7329 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.000446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.001355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.001639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.001973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.006318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:25.007904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:25.009946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 77/158] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:24 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:25.015787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.042107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.065939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.197381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3c5a8-7c8c-4187-88a9-9b1151eb7329 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.239433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992b825-6915-4dc8-9b79-f58b55d0224b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.268408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.269630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.270109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.270589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.294108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:25.310796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:25.311759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:25.312528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:25.316520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 82/159] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:25.320529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.321468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.321947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.322467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.323589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.325051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=d024a945bdaf4746aa6cddd29acc851e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:25.325918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:25.329321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:25.344112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 78/160] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/role_assignments?user.id=d024a945bdaf4746aa6cddd29acc851e&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:25.440022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.479177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c2d28-5244-4c36-ab67-89a8ef684976 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.525772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407d219-de49-4bcd-b61c-16cba36d16e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.526822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d219-de49-4bcd-b61c-16cba36d16e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.527302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d219-de49-4bcd-b61c-16cba36d16e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.527801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d219-de49-4bcd-b61c-16cba36d16e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.528834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407d219-de49-4bcd-b61c-16cba36d16e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:25.529573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407d219-de49-4bcd-b61c-16cba36d16e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:25.531186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407d219-de49-4bcd-b61c-16cba36d16e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:25.545956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 83/161] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:25.551900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.610841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c7ede-93d4-450f-b136-53a3cb756813 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.611432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c7ede-93d4-450f-b136-53a3cb756813 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.611902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c7ede-93d4-450f-b136-53a3cb756813 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.613549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 84/162] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:25.619854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa82850-c9d7-4d14-9f20-cd7c8afd80bf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.620526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa82850-c9d7-4d14-9f20-cd7c8afd80bf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.621029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa82850-c9d7-4d14-9f20-cd7c8afd80bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.648211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6fa82850-c9d7-4d14-9f20-cd7c8afd80bf None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:25.650606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6fa82850-c9d7-4d14-9f20-cd7c8afd80bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:25.717950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.745930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924318d7-d8c3-43a7-a53f-b96e9473c6e1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.801851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 85/163] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:25.817949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb3a4d-8eed-47d0-9ca1-5117b2fe3bc3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.819992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb3a4d-8eed-47d0-9ca1-5117b2fe3bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.820506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb3a4d-8eed-47d0-9ca1-5117b2fe3bc3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.825863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 86/164] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:25.832386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a344c31-01a8-444f-8827-89d1718c3c9b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.832975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a344c31-01a8-444f-8827-89d1718c3c9b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.833507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a344c31-01a8-444f-8827-89d1718c3c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.850263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.851236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.851313 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.851601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.854129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:25.855186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:25.857700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 79/165] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:25.860255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1a344c31-01a8-444f-8827-89d1718c3c9b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:25.861780 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1a344c31-01a8-444f-8827-89d1718c3c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:25.861954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.978039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.989965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 87/166] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:25.990944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a344c31-01a8-444f-8827-89d1718c3c9b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.084984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811e75a-652d-41bc-b9d6-79952f2c1290 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.158831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.159569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.159847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.160116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.186217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:26.194099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:26.194998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:26.195966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 80/167] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:26.196602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.209550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a344c31-01a8-444f-8827-89d1718c3c9b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.251720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:26.283618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.284279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.284558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.284850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.394800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c7a14-1f7e-4249-82d0-df6a20f898a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.432992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.433808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.434090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.434547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.446945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:26.452725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:26.461094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:26.462041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:26.462944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 81/168] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:26.463616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.465880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:26.467934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:26.602571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 88/169] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:26.603517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.713948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:26.715950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:26.721336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904373d-1dc0-4963-b550-301b6fc140a0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.760394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.761250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.761537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.761813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.762732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:26.763182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:26.764590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:26.775059 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 89/170] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:26.775917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.791066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaf559-8443-4dc5-aa20-cb264baa5e8f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.859774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.860478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.860769 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.861079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.861858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:26.862169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:26.863671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:26.876018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 82/171] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:26.882309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.916194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36811445-fb4c-44e0-b91e-6ec7fd548d7e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.953756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0a5d5-62f9-4c43-9ffc-a196524d0c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.954628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0a5d5-62f9-4c43-9ffc-a196524d0c34 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.954922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0a5d5-62f9-4c43-9ffc-a196524d0c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.955203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0a5d5-62f9-4c43-9ffc-a196524d0c34 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.956524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0a5d5-62f9-4c43-9ffc-a196524d0c34 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:26.958128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0a5d5-62f9-4c43-9ffc-a196524d0c34 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:26.964941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 90/172] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:26 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:26.964941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d7305-0e55-4ceb-8f7d-6301756141bc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.964941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d7305-0e55-4ceb-8f7d-6301756141bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.964941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d7305-0e55-4ceb-8f7d-6301756141bc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.964941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 91/173] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:26.964941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10494df-3a92-414c-ade1-301230622841 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.965368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10494df-3a92-414c-ade1-301230622841 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.965368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10494df-3a92-414c-ade1-301230622841 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.966511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 92/174] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:26.968818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10494df-3a92-414c-ade1-301230622841 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.996575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d8b4-fc31-42ee-83a3-60ab3fa0d091 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.003661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.031237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.032114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.032397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.032671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] PATH_INFO: `/v3/users/d024a945bdaf4746aa6cddd29acc851e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.033907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] RBAC: Authorizing `identity:get_user(user_id=d024a945bdaf4746aa6cddd29acc851e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:27.035378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:27.037230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 83/175] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:26 2022] GET /identity/v3/users/d024a945bdaf4746aa6cddd29acc851e => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:27.038003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.047237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.082444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10494df-3a92-414c-ade1-301230622841 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.117932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb670-b174-471b-a9a2-4b1a8a36e4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.118816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb670-b174-471b-a9a2-4b1a8a36e4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.119098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb670-b174-471b-a9a2-4b1a8a36e4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.119374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb670-b174-471b-a9a2-4b1a8a36e4f0 None admin] PATH_INFO: `/v3/users/55302a4556e140cfa7314d27f1f40743` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.120587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bb670-b174-471b-a9a2-4b1a8a36e4f0 None admin] RBAC: Authorizing `identity:get_user(user_id=55302a4556e140cfa7314d27f1f40743)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:27.122240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bb670-b174-471b-a9a2-4b1a8a36e4f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:27.123764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 93/176] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:26 2022] GET /identity/v3/users/55302a4556e140cfa7314d27f1f40743 => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:27.123764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10fd9a-4831-42e7-967e-0a7920dd9127 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.123764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10fd9a-4831-42e7-967e-0a7920dd9127 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.123764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10fd9a-4831-42e7-967e-0a7920dd9127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.141985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8b48b-80a5-495e-a508-b006e7d3f765 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.143136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c10fd9a-4831-42e7-967e-0a7920dd9127 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:27.144327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c10fd9a-4831-42e7-967e-0a7920dd9127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:27.178303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf337fa-fa99-4173-8151-a5f93cc45f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.178971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf337fa-fa99-4173-8151-a5f93cc45f35 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.179259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf337fa-fa99-4173-8151-a5f93cc45f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.179536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf337fa-fa99-4173-8151-a5f93cc45f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.180381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf337fa-fa99-4173-8151-a5f93cc45f35 None admin] RBAC: Adding query filter params (role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=d024a945bdaf4746aa6cddd29acc851e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:27.180692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf337fa-fa99-4173-8151-a5f93cc45f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:27.182184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf337fa-fa99-4173-8151-a5f93cc45f35 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:27.188116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 84/177] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/role_assignments?user.id=d024a945bdaf4746aa6cddd29acc851e&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=4f582d9851794ff894a232e045525ec4 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:27.189316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df643e-d9d5-47cd-a129-b153d97a8dbe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.189616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df643e-d9d5-47cd-a129-b153d97a8dbe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.189898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df643e-d9d5-47cd-a129-b153d97a8dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.216511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 94/178] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:27.220231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e849-cbd7-4543-9945-c084aa03c1af None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.220524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e849-cbd7-4543-9945-c084aa03c1af None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.220842 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e849-cbd7-4543-9945-c084aa03c1af None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.222141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d2df643e-d9d5-47cd-a129-b153d97a8dbe None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:27.222658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 95/179] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:27.223440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e849-cbd7-4543-9945-c084aa03c1af None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.225274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d2df643e-d9d5-47cd-a129-b153d97a8dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:27.257115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.311827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.318188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 85/180] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:27.319244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df643e-d9d5-47cd-a129-b153d97a8dbe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.366262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e849-cbd7-4543-9945-c084aa03c1af None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.410311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.410979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.411290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.411567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.438436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df643e-d9d5-47cd-a129-b153d97a8dbe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.473201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de0ef5-1da2-480a-b05e-d45c8fbaa541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.474004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de0ef5-1da2-480a-b05e-d45c8fbaa541 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.474810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de0ef5-1da2-480a-b05e-d45c8fbaa541 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.475219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de0ef5-1da2-480a-b05e-d45c8fbaa541 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.476453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de0ef5-1da2-480a-b05e-d45c8fbaa541 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:27.477961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de0ef5-1da2-480a-b05e-d45c8fbaa541 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:27.479672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 86/181] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:27.480838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aae4c0-0b5f-47fd-a19e-1870ae080e1e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.481127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aae4c0-0b5f-47fd-a19e-1870ae080e1e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.481405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aae4c0-0b5f-47fd-a19e-1870ae080e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.487130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:27.488718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:27.498330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d0aae4c0-0b5f-47fd-a19e-1870ae080e1e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:27.499520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d0aae4c0-0b5f-47fd-a19e-1870ae080e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:27.514666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.562524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.567762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 87/182] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:27.580975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aae4c0-0b5f-47fd-a19e-1870ae080e1e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.592536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 96/183] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:27.593299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.671892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aae4c0-0b5f-47fd-a19e-1870ae080e1e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.696890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c06e4-2f83-42af-aa37-dd6b9ade1ad6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.708176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.710120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.710488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.710710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.721787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:27.731415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:27.732314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:27.733142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:27.736917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 88/184] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:27.738048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.746571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.757433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.758236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.758519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.758793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.768945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:27.777622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:27.778712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:27.779494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 97/185] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:27.780137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.799215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:27.841468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedd9f4-b2f7-422a-b056-6bb526ba006e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.876157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.877418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.877418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.877674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/55302a4556e140cfa7314d27f1f40743/roles/21ae2370a18d4c9ebbd8a56f38d6b550` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.879935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] RBAC: Authorizing `identity:create_grant(project_id=8efd660d626b4141b5c9be1824e7e416, user_id=55302a4556e140cfa7314d27f1f40743, role_id=21ae2370a18d4c9ebbd8a56f38d6b550)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:27.881583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:27.889566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5521d-dad4-47f1-9bcd-dd5c45c2f9bb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:27.925980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:27.929155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:27.929155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:27.929155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:27.929155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:27.930147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:27.931776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 98/186] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:27.932548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.960630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 89/187] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:27 2022] PUT /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416/users/55302a4556e140cfa7314d27f1f40743/roles/21ae2370a18d4c9ebbd8a56f38d6b550 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:27.961510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:27.994613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.057713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea0db0-a613-40ea-8f52-e7299ab60f1d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.100457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.101370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.101505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dac82-89aa-4a18-b69f-3ae26e554b29 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.101692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.101925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] PATH_INFO: `/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.103231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=1a5b6641f18e4b10ac87673e3eee0ebb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.104761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.111718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 99/188] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/projects/1a5b6641f18e4b10ac87673e3eee0ebb => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.112976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.122822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.143753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.144429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.144714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.145104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.145866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:28.146226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.148331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.159237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 90/189] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.160092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.234917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430e985-e726-4cca-bbd6-51862faa9aa4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.257623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.272194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.275797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.275797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.275797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.276136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f29a-5dac-44ae-ba4d-4daecc71567d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.276223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:28.276223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.276223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.284947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 100/190] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.290189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.312140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.312904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.313141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.313421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.314503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.315677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.402239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cd02c-c812-436f-9082-decd58c0bc9c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.437954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 91/191] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:28 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:28.438809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.449175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.450262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.450781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.451263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.452350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=1a5b6641f18e4b10ac87673e3eee0ebb, user.id=d024a945bdaf4746aa6cddd29acc851e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:28.452918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.454740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.463502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 101/192] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/role_assignments?user.id=d024a945bdaf4746aa6cddd29acc851e&scope.project.id=1a5b6641f18e4b10ac87673e3eee0ebb&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.464933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.515497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.541242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.603454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdf4c-6e42-4539-842c-8ad7a74c6a9d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.654016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.654964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.655250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.655526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] PATH_INFO: `/v3/users/1a611398a384446d88aa936ce8c3b08e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.661322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b341-6c12-4951-8dfe-8ee3c8d8af7a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.688255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] RBAC: Authorizing `identity:get_user(user_id=1a611398a384446d88aa936ce8c3b08e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.689873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.691787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 102/193] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/users/1a611398a384446d88aa936ce8c3b08e => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.698665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.731446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.732395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.732671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.733030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] PATH_INFO: `/v3/users/1a611398a384446d88aa936ce8c3b08e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.734790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] RBAC: Authorizing `identity:get_user(user_id=1a611398a384446d88aa936ce8c3b08e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.736292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.738294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 92/194] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/users/1a611398a384446d88aa936ce8c3b08e => generated 283 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.746804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.861745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952f032-cb58-444c-9cd1-cf20dc133ade None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.864220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.890557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08014d1d-772c-452d-a06c-012ea536c23d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.894992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.922573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.923276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.923567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.924179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.941283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.942603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.943808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.944345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:29.020812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:29.023338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:29.038786 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:29.041200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:29.193127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 103/195] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:24:29.203623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:29.210310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:29.215596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 93/196] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:24:29.222429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:29.226463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:29.342448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754baec-be4f-4295-a5c8-890c8682d06a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:29.382911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:29.383830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:29.384115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:29.384390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:29.385949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:29.387459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:29.389589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 104/197] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:29 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:29.395668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:29.475420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:29.527158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456bd91-2584-44d5-8c29-720d4d6bdbe1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:29.530365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22000-c64c-4ae8-ac0a-efee9da93f04 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:29.607204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:29.608227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151b90d-1f0b-49c7-a894-612440809741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:29.609203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:29.610303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151b90d-1f0b-49c7-a894-612440809741 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:29.610789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:29.611318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151b90d-1f0b-49c7-a894-612440809741 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:29.611794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:29.612269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151b90d-1f0b-49c7-a894-612440809741 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:29.613406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=1a611398a384446d88aa936ce8c3b08e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:29.613922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:29.616276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:29.621625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151b90d-1f0b-49c7-a894-612440809741 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:29.623056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 105/198] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:29 2022] GET /identity/v3/role_assignments?user.id=1a611398a384446d88aa936ce8c3b08e&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:29.623164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151b90d-1f0b-49c7-a894-612440809741 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:29.625098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 94/199] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:29 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:29.631881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:29.801498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f83ff-c748-4275-a4a9-92c39f475d05 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:29.844298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02296c-3042-4b80-8803-8c9aceea2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:29.844991 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02296c-3042-4b80-8803-8c9aceea2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:29.845280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02296c-3042-4b80-8803-8c9aceea2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:29.845595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02296c-3042-4b80-8803-8c9aceea2cf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:29.846453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02296c-3042-4b80-8803-8c9aceea2cf5 None admin] RBAC: Adding query filter params (role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=1a611398a384446d88aa936ce8c3b08e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:29.846772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02296c-3042-4b80-8803-8c9aceea2cf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:29.848298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02296c-3042-4b80-8803-8c9aceea2cf5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:29.851636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:29.860110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 106/200] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:29 2022] GET /identity/v3/role_assignments?user.id=1a611398a384446d88aa936ce8c3b08e&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=4f582d9851794ff894a232e045525ec4 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:30.396913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af578-69f0-4b00-b697-a37688ad1f07 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:30.397938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af578-69f0-4b00-b697-a37688ad1f07 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:30.398420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af578-69f0-4b00-b697-a37688ad1f07 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:30.400273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 95/201] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:30.405667 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07149-c113-4e13-98ac-8a3489af5337 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:30.406475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07149-c113-4e13-98ac-8a3489af5337 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:30.406961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07149-c113-4e13-98ac-8a3489af5337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:30.437777 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2cd07149-c113-4e13-98ac-8a3489af5337 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:30.439288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2cd07149-c113-4e13-98ac-8a3489af5337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:30.542971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 107/202] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:30.579553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2af578-69f0-4b00-b697-a37688ad1f07 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:30.693875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:30.783609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2af578-69f0-4b00-b697-a37688ad1f07 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:30.838977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e55b4c-f527-4ca9-bb6b-52b3287c4fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:30.839799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e55b4c-f527-4ca9-bb6b-52b3287c4fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:30.840082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e55b4c-f527-4ca9-bb6b-52b3287c4fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:30.840355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e55b4c-f527-4ca9-bb6b-52b3287c4fb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:30.859902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e55b4c-f527-4ca9-bb6b-52b3287c4fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c7e55b4c-f527-4ca9-bb6b-52b3287c4fb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:30.868221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:30.869145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:30.870052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 96/203] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:30.877114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd07149-c113-4e13-98ac-8a3489af5337 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.099599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd07149-c113-4e13-98ac-8a3489af5337 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.134208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761bc61-e72b-448f-85d8-751455e1777b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.134525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761bc61-e72b-448f-85d8-751455e1777b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.134775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761bc61-e72b-448f-85d8-751455e1777b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.136426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 97/204] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:31.142015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9086-4bec-4ad9-9e60-bb36ea3bf3cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.142306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9086-4bec-4ad9-9e60-bb36ea3bf3cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.142588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9086-4bec-4ad9-9e60-bb36ea3bf3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.148230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.154331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.155001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.155283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.155557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.156313 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:31.156616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.158049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.173183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7eba9086-4bec-4ad9-9e60-bb36ea3bf3cf None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:31.174390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7eba9086-4bec-4ad9-9e60-bb36ea3bf3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:31.179447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 108/205] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:31.185209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.313512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 98/206] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:31.315331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9af25-14c1-408f-a5ea-d0ca7c18e1dd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.315836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9af25-14c1-408f-a5ea-d0ca7c18e1dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.316451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9af25-14c1-408f-a5ea-d0ca7c18e1dd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.322376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 99/207] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:31.323934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769f5ce-87b6-4dc4-899c-77749fcbaba6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.324424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769f5ce-87b6-4dc4-899c-77749fcbaba6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.324993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769f5ce-87b6-4dc4-899c-77749fcbaba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.329164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d6c9-3b5c-47c0-beb9-742f8f208018 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.343800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7769f5ce-87b6-4dc4-899c-77749fcbaba6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:31.345328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7769f5ce-87b6-4dc4-899c-77749fcbaba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:31.368524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.369127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.369414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.369690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] PATH_INFO: `/v3/users/55302a4556e140cfa7314d27f1f40743` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.370926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] RBAC: Authorizing `identity:get_user(user_id=55302a4556e140cfa7314d27f1f40743)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.372416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.374489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 109/208] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/users/55302a4556e140cfa7314d27f1f40743 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:31.379712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.415691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.432802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 100/209] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:31.432802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769f5ce-87b6-4dc4-899c-77749fcbaba6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.484861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.543520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f49b1-1eaa-4206-a80d-0c28e1750d19 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.560896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769f5ce-87b6-4dc4-899c-77749fcbaba6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.595309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.596110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.596394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.596669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.606654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.608031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.608539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.609063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.610159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:31.619498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:31.620418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:31.621253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 110/210] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:31.621919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.692336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.703846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.705976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.752692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54beec4-acca-4ac0-ad30-3a7e21abe1c4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.780549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.795957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.796768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.797095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.797375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.803511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.803511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.809526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 111/211] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:31.810580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.852226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 101/212] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:31.852226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.924500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423a2e-845f-4025-be41-029c2144b533 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.970114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.975288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.976397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.977264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.983160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.983160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:31.983160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.983383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb9ecc-4c14-4dea-b562-4443ac5581e6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.983455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.990795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 112/213] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:31.993115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.019270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.019949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.020321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.020595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.022146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.023538 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:32.049018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.100841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce7c1-7dca-4837-9ad1-a6e526d8a5b9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.133835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0b8f5-6a47-4978-a802-24fd0bd209f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.134706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0b8f5-6a47-4978-a802-24fd0bd209f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.135018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0b8f5-6a47-4978-a802-24fd0bd209f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.135297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0b8f5-6a47-4978-a802-24fd0bd209f5 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.136474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0b8f5-6a47-4978-a802-24fd0bd209f5 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.137973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0b8f5-6a47-4978-a802-24fd0bd209f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:32.139669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 113/214] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:32.140950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368f9ec-654a-425a-9f81-124ade729cf6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.141236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368f9ec-654a-425a-9f81-124ade729cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.141512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368f9ec-654a-425a-9f81-124ade729cf6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.142901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 114/215] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:32.143487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368f9ec-654a-425a-9f81-124ade729cf6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.172835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 102/216] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:31 2022] POST /identity/v3/users => generated 297 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:32.173683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.298203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368f9ec-654a-425a-9f81-124ade729cf6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.308057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3669803-68fe-489d-8d8e-475611393e53 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.325605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.348278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.349459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.349770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.350234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] PATH_INFO: `/v3/users/a822e4a1e8fd444fa7c5924de6dc49de` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.350951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff304959-61c2-481a-bfee-593ca704dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.351648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff304959-61c2-481a-bfee-593ca704dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.351968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff304959-61c2-481a-bfee-593ca704dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.352260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff304959-61c2-481a-bfee-593ca704dbe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.353398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff304959-61c2-481a-bfee-593ca704dbe1 None admin] RBAC: Adding query filter params (role.id=21ae2370a18d4c9ebbd8a56f38d6b550, scope.project.id=8efd660d626b4141b5c9be1824e7e416, user.id=55302a4556e140cfa7314d27f1f40743) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:32.353769 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff304959-61c2-481a-bfee-593ca704dbe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.355364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff304959-61c2-481a-bfee-593ca704dbe1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:32.363581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 103/217] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:32 2022] GET /identity/v3/role_assignments?user.id=55302a4556e140cfa7314d27f1f40743&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=21ae2370a18d4c9ebbd8a56f38d6b550 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:32.366046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf5ed-c546-4477-9035-9b7b21886128 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.366046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf5ed-c546-4477-9035-9b7b21886128 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.366046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf5ed-c546-4477-9035-9b7b21886128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.372080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] RBAC: Authorizing `identity:get_user(user_id=a822e4a1e8fd444fa7c5924de6dc49de)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.373986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:32.376841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 115/218] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:32 2022] GET /identity/v3/users/a822e4a1e8fd444fa7c5924de6dc49de => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:32.383510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.389251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-312cf5ed-c546-4477-9035-9b7b21886128 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:32.390953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-312cf5ed-c546-4477-9035-9b7b21886128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:32.401201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.483934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 104/219] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:32.486129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0a24c-a246-4f22-8d29-d7c7748bc63d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.486419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0a24c-a246-4f22-8d29-d7c7748bc63d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.486694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0a24c-a246-4f22-8d29-d7c7748bc63d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.488297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 105/220] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:32.499434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97d811-5a0c-45b1-9d6b-bd117c2e00b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.499729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97d811-5a0c-45b1-9d6b-bd117c2e00b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.500006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97d811-5a0c-45b1-9d6b-bd117c2e00b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.517498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ee97d811-5a0c-45b1-9d6b-bd117c2e00b9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:32.518706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee97d811-5a0c-45b1-9d6b-bd117c2e00b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:32.530232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f7638-e79e-42e5-8957-9e57a3f28855 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.576477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.577425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.577859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.578284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.613903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 106/221] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:32.614796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97d811-5a0c-45b1-9d6b-bd117c2e00b9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.650566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.681348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.683225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:32.701769 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.732364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97d811-5a0c-45b1-9d6b-bd117c2e00b9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.781276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.782084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.782361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.782629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.800371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:32.810470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:32.811346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:32.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:32.814192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 107/222] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:32.815509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.837022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 116/223] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:32.838405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.925402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.937619 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fa006-db37-471b-8753-de550c162eb1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.954127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd407e-5250-4625-a66c-10d12d1ea96c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.977225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.984000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.985334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.985905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.986333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.995063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.995940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.996380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.996718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.997595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:32.997999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.999489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.001564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:33.011932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:33.012870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:33.013689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:33.014814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 117/224] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.015781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.016365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 108/225] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:33.017373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.138860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fd4d7-0339-4f2c-98d8-982f7e459348 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.147341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b417278-f822-4383-a47b-d6947aaf1728 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.182701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.183683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.184044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.184164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.184386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] PATH_INFO: `/v3/users/1a611398a384446d88aa936ce8c3b08e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.185523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.185710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] RBAC: Authorizing `identity:get_user(user_id=1a611398a384446d88aa936ce8c3b08e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.186174 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.186658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.187446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.188327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.190070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.192263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 118/226] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:33 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.193353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 109/227] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:33 2022] GET /identity/v3/users/1a611398a384446d88aa936ce8c3b08e => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.194035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.195812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.205651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:33.269200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:33.313044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d73c1-33bb-40d4-99fa-11e1fe02ed01 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.317995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633700e4-0b31-4660-97f8-5a4d98400867 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.350895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.351733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.352085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.352433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.353287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:33.353661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.355134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.356076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.357170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.357639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.358181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.359621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.361321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.363244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 110/228] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:33 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.364570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.369792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 119/229] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.370491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.445646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:33.473152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9852f37-fdc1-401d-8bec-1300229a63d7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.481934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1b75a-6839-48b5-9528-8062003f7380 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.509171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:33.514029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.515352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.515865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.516739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.518262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=a822e4a1e8fd444fa7c5924de6dc49de) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:33.518865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.521390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.522248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.523166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.523443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.523714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/1a611398a384446d88aa936ce8c3b08e/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.525917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6581cd5cb54c7aacb51e90736e34aa, user_id=1a611398a384446d88aa936ce8c3b08e, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.527435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.529279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 111/230] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:33 2022] GET /identity/v3/role_assignments?user.id=a822e4a1e8fd444fa7c5924de6dc49de&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.530561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.653027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 120/231] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:33 2022] PUT /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/1a611398a384446d88aa936ce8c3b08e/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:33.660871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e56b-3e3d-4b33-8e89-f97ddcc5a34a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.704374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.705583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.706070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.706528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] PATH_INFO: `/v3/users/1a611398a384446d88aa936ce8c3b08e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.708486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] RBAC: Authorizing `identity:get_user(user_id=1a611398a384446d88aa936ce8c3b08e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.710171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.712475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 112/232] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:33 2022] GET /identity/v3/users/1a611398a384446d88aa936ce8c3b08e => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.717860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.838067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a878980-dd4a-490e-9e31-cb1bbaca1e13 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:33.867710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:33.876132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148d95e-4ada-4d5f-866a-d55a22578f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:33.877398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148d95e-4ada-4d5f-866a-d55a22578f0a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:33.877686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148d95e-4ada-4d5f-866a-d55a22578f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:33.883614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148d95e-4ada-4d5f-866a-d55a22578f0a None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:33.883614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148d95e-4ada-4d5f-866a-d55a22578f0a None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:33.883614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148d95e-4ada-4d5f-866a-d55a22578f0a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:33.883614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 121/233] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:33 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:33.889196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:33.972834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:34.021186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353abd4-dee8-4271-871e-395c2f39b3a9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:34.060232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bed9cf-e6a5-44e4-b9d3-873b912d12f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:34.061440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bed9cf-e6a5-44e4-b9d3-873b912d12f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.061878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bed9cf-e6a5-44e4-b9d3-873b912d12f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.062328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bed9cf-e6a5-44e4-b9d3-873b912d12f2 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/1a611398a384446d88aa936ce8c3b08e/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.064742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bed9cf-e6a5-44e4-b9d3-873b912d12f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6581cd5cb54c7aacb51e90736e34aa, user_id=1a611398a384446d88aa936ce8c3b08e, role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:34.066388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bed9cf-e6a5-44e4-b9d3-873b912d12f2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:34.170418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 113/234] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:33 2022] PUT /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/1a611398a384446d88aa936ce8c3b08e/roles/4f582d9851794ff894a232e045525ec4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:34.711560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907446e-a2aa-416d-83e8-3940cffc94ef None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.712107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907446e-a2aa-416d-83e8-3940cffc94ef None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.712107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907446e-a2aa-416d-83e8-3940cffc94ef None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.713778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 122/235] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:34.719207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e9e4-3b5b-4fb6-a93d-240d7f191784 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.720065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e9e4-3b5b-4fb6-a93d-240d7f191784 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.720561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e9e4-3b5b-4fb6-a93d-240d7f191784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.739000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2638e9e4-3b5b-4fb6-a93d-240d7f191784 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:34.741477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2638e9e4-3b5b-4fb6-a93d-240d7f191784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:34.760540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55bf3f-b466-49cc-98b4-5648628edf36 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.761131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55bf3f-b466-49cc-98b4-5648628edf36 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.761510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55bf3f-b466-49cc-98b4-5648628edf36 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.763073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 123/236] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:34.768907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b6139-2790-44ef-aea8-7ed54ab13ad3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.769447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b6139-2790-44ef-aea8-7ed54ab13ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.769943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b6139-2790-44ef-aea8-7ed54ab13ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.798930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-682b6139-2790-44ef-aea8-7ed54ab13ad3 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:34.800182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-682b6139-2790-44ef-aea8-7ed54ab13ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:34.868598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 114/237] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:34.918223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e9e4-3b5b-4fb6-a93d-240d7f191784 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:34.921505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 124/238] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:34.968226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b6139-2790-44ef-aea8-7ed54ab13ad3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:34.989356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:35.039840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e9e4-3b5b-4fb6-a93d-240d7f191784 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:35.079462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:35.080262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.080550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.080850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.083876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:35.085393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:35.087042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 115/239] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:35.093457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:35.164407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b6139-2790-44ef-aea8-7ed54ab13ad3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:35.218984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:35.236969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:35.237776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.238060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.238336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.252745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:35.262278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:35.263276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:35.264060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 125/240] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:35.289958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9aca2-ab98-4217-88f8-983d6c2e56c9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:35.303996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:35.312259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:35.335452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:35.336573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.337241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.337715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.338627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:35.340414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:35.428023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 116/241] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:35 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:35.471760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09094e39-6a9b-45ba-af3c-24b770282392 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:35.512753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:35.513904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.514642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.515111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.516228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:35.516836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:35.518484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:35.529606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 126/242] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:35.536679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:35.655947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b18f-efd2-4034-a150-178143403042 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:35.699464 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe309e9-957c-42c0-9961-20814cdb8b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:35.700711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe309e9-957c-42c0-9961-20814cdb8b5e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.701287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe309e9-957c-42c0-9961-20814cdb8b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.701767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe309e9-957c-42c0-9961-20814cdb8b5e None admin] PATH_INFO: `/v3/users/00e1244f84e84cbd876ef7ccdc632676` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.718867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe309e9-957c-42c0-9961-20814cdb8b5e None admin] RBAC: Authorizing `identity:get_user(user_id=00e1244f84e84cbd876ef7ccdc632676)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:35.720759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe309e9-957c-42c0-9961-20814cdb8b5e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:35.724553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 117/243] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:35 2022] GET /identity/v3/users/00e1244f84e84cbd876ef7ccdc632676 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:35.730621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:35.788265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:35.845197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9b1d-c7a5-4ec5-971e-13e7acbf242a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:35.881968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:35.882630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.882916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.883197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.942388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442767c0-1ecf-4cf7-a805-18707e9f9fd6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.942720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442767c0-1ecf-4cf7-a805-18707e9f9fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.942960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442767c0-1ecf-4cf7-a805-18707e9f9fd6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.945062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 118/244] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:35.950455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546eeb3-cec9-4449-b85b-a80b1ab96869 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.950929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546eeb3-cec9-4449-b85b-a80b1ab96869 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.951218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546eeb3-cec9-4449-b85b-a80b1ab96869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.967420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:35.969273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:35.972643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7546eeb3-cec9-4449-b85b-a80b1ab96869 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:35.975705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7546eeb3-cec9-4449-b85b-a80b1ab96869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:36.006434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:36.039871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:36.065347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 119/245] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:36.094733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546eeb3-cec9-4449-b85b-a80b1ab96869 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:36.099953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 127/246] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:36.109680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:36.215721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546eeb3-cec9-4449-b85b-a80b1ab96869 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:36.215909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259af04-aa4b-4bf9-906a-690f26e08f9f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:36.253390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:36.254453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.255347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.255607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.256992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:36.258558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:36.259286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:36.260108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.260554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 128/247] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:36 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:36.260714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.260714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.266125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:36.281493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:36.289842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:36.290737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:36.292117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 120/248] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:36.297394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:36.378287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39afb0-67f5-4d74-98f4-0d5b25b731f9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:36.412481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b97394-057a-4094-beea-d14b807527a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:36.417348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:36.418029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.418316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.418596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.419509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] RBAC: Adding query filter params (role.id=21ae2370a18d4c9ebbd8a56f38d6b550, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=00e1244f84e84cbd876ef7ccdc632676) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:36.419889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:36.421966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:36.429086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 129/249] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:36 2022] GET /identity/v3/role_assignments?user.id=00e1244f84e84cbd876ef7ccdc632676&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=21ae2370a18d4c9ebbd8a56f38d6b550 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:36.452947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0213cac-9b56-4653-a74a-bf8ade7870a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:36.453649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0213cac-9b56-4653-a74a-bf8ade7870a5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.453938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0213cac-9b56-4653-a74a-bf8ade7870a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.454216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0213cac-9b56-4653-a74a-bf8ade7870a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.455003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0213cac-9b56-4653-a74a-bf8ade7870a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:36.455314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0213cac-9b56-4653-a74a-bf8ade7870a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:36.456815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0213cac-9b56-4653-a74a-bf8ade7870a5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:36.472324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 121/250] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:36.474362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:36.501176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:36.517606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11ab3-af94-4859-ba00-4ea72f88998c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.517898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11ab3-af94-4859-ba00-4ea72f88998c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.518073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11ab3-af94-4859-ba00-4ea72f88998c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.519651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 122/251] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:36.524354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:36.528833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d375d-715c-42ee-a5b6-5c272c07fba1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.528833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d375d-715c-42ee-a5b6-5c272c07fba1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.528833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d375d-715c-42ee-a5b6-5c272c07fba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.547838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-580d375d-715c-42ee-a5b6-5c272c07fba1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:36.549109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-580d375d-715c-42ee-a5b6-5c272c07fba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:36.630490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:36.652339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 123/252] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:36.677846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878afa8e-271f-401f-b2e7-1a71fa89673d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:36.686955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8091-fa33-422b-9f8d-cc194d475ce4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.687251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8091-fa33-422b-9f8d-cc194d475ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.687531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8091-fa33-422b-9f8d-cc194d475ce4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.689101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 124/253] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:36.694672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8091-fa33-422b-9f8d-cc194d475ce4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:36.720119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:36.728317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7e870-ffbb-4eb3-af9a-ac3ea98cd7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:36.729304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7e870-ffbb-4eb3-af9a-ac3ea98cd7be None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.729726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7e870-ffbb-4eb3-af9a-ac3ea98cd7be None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.729939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7e870-ffbb-4eb3-af9a-ac3ea98cd7be None admin] PATH_INFO: `/v3/users/a822e4a1e8fd444fa7c5924de6dc49de` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.731185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7e870-ffbb-4eb3-af9a-ac3ea98cd7be None admin] RBAC: Authorizing `identity:get_user(user_id=a822e4a1e8fd444fa7c5924de6dc49de)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:36.732749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7e870-ffbb-4eb3-af9a-ac3ea98cd7be None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:36.734838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 130/254] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:36 2022] GET /identity/v3/users/a822e4a1e8fd444fa7c5924de6dc49de => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:36.736150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79a6f-b0de-47aa-a0e8-954a16e9383f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.736438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79a6f-b0de-47aa-a0e8-954a16e9383f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.736718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79a6f-b0de-47aa-a0e8-954a16e9383f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.807864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-45f79a6f-b0de-47aa-a0e8-954a16e9383f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:36.808434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-45f79a6f-b0de-47aa-a0e8-954a16e9383f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:36.910914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8091-fa33-422b-9f8d-cc194d475ce4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:36.953963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:36.954779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:36.956334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:36.956617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:36.959810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 131/255] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:36.960443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f79a6f-b0de-47aa-a0e8-954a16e9383f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:36.988347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:37.000272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:37.001227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:37.002001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:37.008197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 125/256] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:37.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.089896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:37.121456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f79a6f-b0de-47aa-a0e8-954a16e9383f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.191262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.192857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.193170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.193449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.194269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:37.195324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:37.196978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:37.198726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 132/257] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:36 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:37.199554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.233764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a2f25-b6cf-4b12-8f30-5875db511428 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.269616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.271269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.271763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.272209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.282974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:37.292105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:37.293021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:37.293797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:37.296643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 126/258] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:37.297609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.404404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e1d3c-fae0-41be-811c-e702a5285bdc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.414444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe28fd-e156-4c52-9599-6ca13cf8310f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.440684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.442185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.443100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.443598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/a822e4a1e8fd444fa7c5924de6dc49de/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.445871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6581cd5cb54c7aacb51e90736e34aa, user_id=a822e4a1e8fd444fa7c5924de6dc49de, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:37.447592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:37.452271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:37.485051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.486062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.486543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.487016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.488088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:37.488635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:37.490985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:37.500023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 133/259] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:37.501262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.513453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:37.563224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 127/260] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:37 2022] PUT /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/a822e4a1e8fd444fa7c5924de6dc49de/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:37.564571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.669024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84152b45-4877-4320-9f9d-2ee46fda01d4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.745282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac709449-c252-44c0-86d9-56d35c282643 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.759695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.761131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.761627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.762123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.763156 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:37.763705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:37.765361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:37.785281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 134/261] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:37.790028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.831336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.832695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.833241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.833732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] PATH_INFO: `/v3/users/1a611398a384446d88aa936ce8c3b08e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.835437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] RBAC: Authorizing `identity:get_user(user_id=1a611398a384446d88aa936ce8c3b08e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:37.839865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:37.843519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 128/262] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:37 2022] GET /identity/v3/users/1a611398a384446d88aa936ce8c3b08e => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:37.850075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.919308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:37.954285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d37795-90ae-4e16-b759-c5e82eeac750 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.984964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:37.992099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.998041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.998041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.998041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] PATH_INFO: `/v3/users/1a611398a384446d88aa936ce8c3b08e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.998041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] RBAC: Authorizing `identity:get_user(user_id=1a611398a384446d88aa936ce8c3b08e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:37.998041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:37.998818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 135/263] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:37 2022] GET /identity/v3/users/1a611398a384446d88aa936ce8c3b08e => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:38.006625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:38.045025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29372cbc-8bbf-47b7-80f6-6478323f5796 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:38.103444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:38.104108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:38.104392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:38.104667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:38.120031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2b8c4-877c-426c-98e4-734dca04aca5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:38.185398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:38.186450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:38.186938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:38.187416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:38.209232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:38.215910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:38.229777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:38.311366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:38.312912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:38.333391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:38.409692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 129/264] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:38.415340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:38.499076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 136/265] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:38.505499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:38.546902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:38.620693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0039f-792b-41f6-a5e2-280a0dfb3409 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:38.687103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a0f39-eaea-468d-a2f7-9ef21cad04eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:38.693799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:38.694725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:38.695902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:38.696181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:38.697524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:38.698996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:38.700754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 130/266] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:38 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:38.708293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:38.778759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:38.779549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:38.779830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:38.780100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:38.781898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:38.783374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:38.785108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 137/267] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:38 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:38.793007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:38.793463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:38.982983 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1873a7-5234-4b5d-abba-aacf6c1b2869 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.037365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb5c05-e61b-47e0-9df2-2bf0f41c114e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.052497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.078389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9ebc3-cba7-4691-9b6d-7d77830ccd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.079082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9ebc3-cba7-4691-9b6d-7d77830ccd97 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.079392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9ebc3-cba7-4691-9b6d-7d77830ccd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.079669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9ebc3-cba7-4691-9b6d-7d77830ccd97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.080558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9ebc3-cba7-4691-9b6d-7d77830ccd97 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=1a611398a384446d88aa936ce8c3b08e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:39.080901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9ebc3-cba7-4691-9b6d-7d77830ccd97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:39.083060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9ebc3-cba7-4691-9b6d-7d77830ccd97 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:39.091400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 131/268] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:38 2022] GET /identity/v3/role_assignments?user.id=1a611398a384446d88aa936ce8c3b08e&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:39.092705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66581e34-93b8-4c8a-a8b6-d8b1eb38da73 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.099514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66581e34-93b8-4c8a-a8b6-d8b1eb38da73 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.099904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66581e34-93b8-4c8a-a8b6-d8b1eb38da73 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.102106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 132/269] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:39.102962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528215-d8c4-4a94-ace6-d7f442af4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.103501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a35869-8b06-40c2-912c-eecf04e6372f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.103789 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a35869-8b06-40c2-912c-eecf04e6372f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.104063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a35869-8b06-40c2-912c-eecf04e6372f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.104593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528215-d8c4-4a94-ace6-d7f442af4dbf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.106718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 133/270] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:39.107390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528215-d8c4-4a94-ace6-d7f442af4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.107895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528215-d8c4-4a94-ace6-d7f442af4dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.109140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528215-d8c4-4a94-ace6-d7f442af4dbf None admin] RBAC: Adding query filter params (role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=1a611398a384446d88aa936ce8c3b08e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:39.109717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528215-d8c4-4a94-ace6-d7f442af4dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:39.111565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528215-d8c4-4a94-ace6-d7f442af4dbf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:39.114821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5473bb-43ff-4647-84b3-210085f79a7a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.115121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5473bb-43ff-4647-84b3-210085f79a7a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.115502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5473bb-43ff-4647-84b3-210085f79a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.124601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 138/271] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:38 2022] GET /identity/v3/role_assignments?user.id=1a611398a384446d88aa936ce8c3b08e&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=4f582d9851794ff894a232e045525ec4 => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:39.126486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51a31-5aaa-4f26-b905-3b5f568cdc38 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.128057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51a31-5aaa-4f26-b905-3b5f568cdc38 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.128823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51a31-5aaa-4f26-b905-3b5f568cdc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.154007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1b5473bb-43ff-4647-84b3-210085f79a7a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:39.155898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b5473bb-43ff-4647-84b3-210085f79a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:39.165326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fcb51a31-5aaa-4f26-b905-3b5f568cdc38 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:39.168145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fcb51a31-5aaa-4f26-b905-3b5f568cdc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:39.327425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 139/272] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:39.329844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e501-5220-4ad8-8afe-a89f2f4d22cc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.331017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e501-5220-4ad8-8afe-a89f2f4d22cc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.332981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e501-5220-4ad8-8afe-a89f2f4d22cc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.335019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 134/273] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:39.335459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 140/274] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:39.351018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1ea9b-ba21-4cfe-8c7e-20ecbb694aa2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.351741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1ea9b-ba21-4cfe-8c7e-20ecbb694aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.352234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1ea9b-ba21-4cfe-8c7e-20ecbb694aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.380482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e501-5220-4ad8-8afe-a89f2f4d22cc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.381801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a4e1ea9b-ba21-4cfe-8c7e-20ecbb694aa2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:39.383080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a4e1ea9b-ba21-4cfe-8c7e-20ecbb694aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:39.413182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.494007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 135/275] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:39.496053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1ea9b-ba21-4cfe-8c7e-20ecbb694aa2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.539689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.544013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e501-5220-4ad8-8afe-a89f2f4d22cc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.586456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.587165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.587450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.587725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.601620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:39.609604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:39.610767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 141/276] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:39.612089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.627754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1ea9b-ba21-4cfe-8c7e-20ecbb694aa2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.666328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.667505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.667794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.668093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.669778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:39.671424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:39.678767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 136/277] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:39.678767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.706518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.727128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0e777-7035-46b6-9819-f477c8becead None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.763471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.764396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.764682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.765168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.777914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:39.786704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:39.788277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 142/278] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:39.789204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.808211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01ad76-60bb-4e8a-9860-03c55dbd54bf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.822577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.845725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.846430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.846720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.847003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.847751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:39.848613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:39.850040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:39.858865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 137/279] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:39.859696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.922483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3cf4e-07b9-463e-85aa-2b901be39cba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.959047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.965111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.965794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.966085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.966586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.967368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:39.968930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:39.976200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555dc98-e315-4b93-b0bb-a82c0a33f0f7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:40.012496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:40.013199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.013509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.013789 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.014539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:40.014852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:40.016231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:40.025664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 138/280] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:40.026489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:40.101490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 143/281] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:39 2022] POST /identity/v3/users => generated 293 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:40.102087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:40.142741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c274-f3cb-4198-a6da-434e8f54eb87 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:40.187988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:40.189419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.189949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.190540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] PATH_INFO: `/v3/users/00e1244f84e84cbd876ef7ccdc632676` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.192870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] RBAC: Authorizing `identity:get_user(user_id=00e1244f84e84cbd876ef7ccdc632676)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:40.194650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:40.197005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 139/282] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:40 2022] GET /identity/v3/users/00e1244f84e84cbd876ef7ccdc632676 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:40.203132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:40.214194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:40.225628 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:40.243692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702625e-0fa0-4420-9752-9270f8a40e55 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:40.284165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:40.285515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.285806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.286127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] PATH_INFO: `/v3/users/6c9ea78355d943849807ec0496725dc7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.303552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] RBAC: Authorizing `identity:get_user(user_id=6c9ea78355d943849807ec0496725dc7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:40.305547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:40.307412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 144/283] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:40 2022] GET /identity/v3/users/6c9ea78355d943849807ec0496725dc7 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:40.315295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:40.333381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6e30c-15e5-4859-9080-05c2fd046605 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:40.374535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e8ebd-cb6a-4617-aa4d-8d2bcae069c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:40.375493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e8ebd-cb6a-4617-aa4d-8d2bcae069c9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.375877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e8ebd-cb6a-4617-aa4d-8d2bcae069c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.376289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e8ebd-cb6a-4617-aa4d-8d2bcae069c9 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.378656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e8ebd-cb6a-4617-aa4d-8d2bcae069c9 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:40.380406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e8ebd-cb6a-4617-aa4d-8d2bcae069c9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:40.382938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 140/284] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:40 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:40.383236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade11c-7ef3-45b1-a49c-d0d864e6a12d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.383526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade11c-7ef3-45b1-a49c-d0d864e6a12d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.383813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade11c-7ef3-45b1-a49c-d0d864e6a12d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.386430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 141/285] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:40.388114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efade11c-7ef3-45b1-a49c-d0d864e6a12d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:40.452700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fa0af-b9ad-47f5-b332-02cf8a6e9c46 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:40.495065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:40.496069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06686070-f99e-4efe-a79f-ce135541057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:40.497190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686070-f99e-4efe-a79f-ce135541057b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.497814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686070-f99e-4efe-a79f-ce135541057b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.498476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686070-f99e-4efe-a79f-ce135541057b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.499891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06686070-f99e-4efe-a79f-ce135541057b None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, user.id=6c9ea78355d943849807ec0496725dc7, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:40.500417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06686070-f99e-4efe-a79f-ce135541057b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:40.502108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06686070-f99e-4efe-a79f-ce135541057b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:40.508896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:40.510205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 145/286] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 18 10:24:40 2022] GET /identity/v3/role_assignments?user.id=6c9ea78355d943849807ec0496725dc7&scope.system=all&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:40.511726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617135-189a-40fe-9115-b9571ecb0a0d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.512212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617135-189a-40fe-9115-b9571ecb0a0d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.512710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617135-189a-40fe-9115-b9571ecb0a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.516838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efade11c-7ef3-45b1-a49c-d0d864e6a12d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:40.546129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b7617135-189a-40fe-9115-b9571ecb0a0d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:40.547775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b7617135-189a-40fe-9115-b9571ecb0a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:40.591409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:40.592855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.593343 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.593811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/00e1244f84e84cbd876ef7ccdc632676/roles/21ae2370a18d4c9ebbd8a56f38d6b550` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.597770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6581cd5cb54c7aacb51e90736e34aa, user_id=00e1244f84e84cbd876ef7ccdc632676, role_id=21ae2370a18d4c9ebbd8a56f38d6b550)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:40.601727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:40.672447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 146/287] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:40.696289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 142/288] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:40 2022] PUT /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/users/00e1244f84e84cbd876ef7ccdc632676/roles/21ae2370a18d4c9ebbd8a56f38d6b550 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:24:40.697678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7617135-189a-40fe-9115-b9571ecb0a0d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:40.854012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7617135-189a-40fe-9115-b9571ecb0a0d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:40.892753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:40.901234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:40.902141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:40.902434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:40.902717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:40.922369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:40.931410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:40.932331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:40.933249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 147/289] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:40.939300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:41.068386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23bb9a-047a-43eb-8ace-a0fef6f2bbe3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:41.134085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:41.134768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:41.135054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:41.135331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:41.136091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:41.136393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:41.137805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:41.153587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 143/290] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:41.162034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:41.288728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ee3c9-ff92-4188-b48c-1534b0b7e093 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:41.334420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:41.335683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:41.336184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:41.336658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] PATH_INFO: `/v3/users/a822e4a1e8fd444fa7c5924de6dc49de` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:41.338106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] RBAC: Authorizing `identity:get_user(user_id=a822e4a1e8fd444fa7c5924de6dc49de)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:41.339845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:41.341744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 148/291] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:41 2022] GET /identity/v3/users/a822e4a1e8fd444fa7c5924de6dc49de => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:41.349988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:41.361863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:41.483349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525b98-2e1f-48a8-99f7-1773b3be0b88 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:41.538436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:41.539171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:41.539425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:41.539705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:41.629734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:41.631338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:41.634129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:41.734129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 144/292] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:41.740385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:41.848422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:41.909617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e6b25-c8cf-473c-91dc-0ed39f761bec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:41.955112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5dbc1-8e73-4ee8-8f11-f8d6eed5d95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:41.955987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5dbc1-8e73-4ee8-8f11-f8d6eed5d95b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:41.956275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5dbc1-8e73-4ee8-8f11-f8d6eed5d95b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:41.956557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5dbc1-8e73-4ee8-8f11-f8d6eed5d95b None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:41.961506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5dbc1-8e73-4ee8-8f11-f8d6eed5d95b None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:41.963035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5dbc1-8e73-4ee8-8f11-f8d6eed5d95b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:41.964971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 149/293] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:41 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:41.971019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:42.087689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e509a4-8012-4613-93b3-9f9a9e65ec3f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:42.099457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:42.124912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5fd4d-7c54-4e84-add7-06c429033ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:42.126008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5fd4d-7c54-4e84-add7-06c429033ffb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.126477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5fd4d-7c54-4e84-add7-06c429033ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.126931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5fd4d-7c54-4e84-add7-06c429033ffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.128315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5fd4d-7c54-4e84-add7-06c429033ffb None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=a822e4a1e8fd444fa7c5924de6dc49de) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:42.128933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5fd4d-7c54-4e84-add7-06c429033ffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:42.131405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5fd4d-7c54-4e84-add7-06c429033ffb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:42.138568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 145/294] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:41 2022] GET /identity/v3/role_assignments?user.id=a822e4a1e8fd444fa7c5924de6dc49de&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:42.677266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf159d7-6104-471d-9acb-8a99d66d7aab None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.677570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf159d7-6104-471d-9acb-8a99d66d7aab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.677839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf159d7-6104-471d-9acb-8a99d66d7aab None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.680405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 150/295] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:42.685919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de369db0-4e7a-41c5-8bc7-24b2a11cdcbb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.686378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de369db0-4e7a-41c5-8bc7-24b2a11cdcbb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.686568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de369db0-4e7a-41c5-8bc7-24b2a11cdcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.705019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-de369db0-4e7a-41c5-8bc7-24b2a11cdcbb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:42.706250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-de369db0-4e7a-41c5-8bc7-24b2a11cdcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:42.785875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 146/296] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:42.809474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf159d7-6104-471d-9acb-8a99d66d7aab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:42.895523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551784a7-1e6e-40cb-b42a-2109ae7a12a8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.896010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551784a7-1e6e-40cb-b42a-2109ae7a12a8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.896559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551784a7-1e6e-40cb-b42a-2109ae7a12a8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.899639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 147/297] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:42.904244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b0b4-75ad-4cf1-84e4-37c0374f2d31 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.904921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b0b4-75ad-4cf1-84e4-37c0374f2d31 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.905464 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b0b4-75ad-4cf1-84e4-37c0374f2d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.924360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0c51b0b4-75ad-4cf1-84e4-37c0374f2d31 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:42.925898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0c51b0b4-75ad-4cf1-84e4-37c0374f2d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:42.932498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf159d7-6104-471d-9acb-8a99d66d7aab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:42.964234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:42.980729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:42.981561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.981848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.982120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.996902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:43.009073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:43.010005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:43.010963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 151/298] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:43.015972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.024963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 148/299] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:43.042041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b0b4-75ad-4cf1-84e4-37c0374f2d31 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.072707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:43.126552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469bac-7467-4a6b-b71c-feb9db2f0773 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.155825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b0b4-75ad-4cf1-84e4-37c0374f2d31 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.161883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa52fc-5120-469c-b34b-0cfa5e4731f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.162554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa52fc-5120-469c-b34b-0cfa5e4731f9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.162881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa52fc-5120-469c-b34b-0cfa5e4731f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.163121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa52fc-5120-469c-b34b-0cfa5e4731f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.163881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa52fc-5120-469c-b34b-0cfa5e4731f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:43.164187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa52fc-5120-469c-b34b-0cfa5e4731f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:43.165678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa52fc-5120-469c-b34b-0cfa5e4731f9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:43.178601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 152/300] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:43.178601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56ef05-2635-4997-a3c9-73a9dff60573 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.178601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56ef05-2635-4997-a3c9-73a9dff60573 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.178601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56ef05-2635-4997-a3c9-73a9dff60573 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.178601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 153/301] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:43.182005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56ef05-2635-4997-a3c9-73a9dff60573 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.199271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1592d-004f-4a0a-ac2c-403d103a7556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.200177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1592d-004f-4a0a-ac2c-403d103a7556 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.200626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1592d-004f-4a0a-ac2c-403d103a7556 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.200936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1592d-004f-4a0a-ac2c-403d103a7556 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.202145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1592d-004f-4a0a-ac2c-403d103a7556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:43.203756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1592d-004f-4a0a-ac2c-403d103a7556 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:43.210978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 149/302] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:43.212368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b29a4c-5f66-4068-9f28-33ee07940874 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.212664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b29a4c-5f66-4068-9f28-33ee07940874 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.213224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b29a4c-5f66-4068-9f28-33ee07940874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.232314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b6b29a4c-5f66-4068-9f28-33ee07940874 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:43.233676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b6b29a4c-5f66-4068-9f28-33ee07940874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:43.234578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:43.295500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56ef05-2635-4997-a3c9-73a9dff60573 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.311577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 150/303] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:43.312852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b29a4c-5f66-4068-9f28-33ee07940874 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.343318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:43.346650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.348619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.348969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.348969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] PATH_INFO: `/v3/users/3fcfbe15957a4b71b896c63baf95775a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.364178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcfbe15957a4b71b896c63baf95775a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:43.365700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:43.367626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 154/304] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/users/3fcfbe15957a4b71b896c63baf95775a => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:43.368481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.424787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b29a4c-5f66-4068-9f28-33ee07940874 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.463158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.463734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.464019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.464296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] PATH_INFO: `/v3/groups` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.465012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:43.466461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:43.477760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf43cc-3aaa-4c71-8916-acbdf0ce6272 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.480041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:43.513303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b2837-ee79-4972-8717-19517d6034b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.514095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2837-ee79-4972-8717-19517d6034b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.514416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2837-ee79-4972-8717-19517d6034b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.514652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2837-ee79-4972-8717-19517d6034b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.525449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b2837-ee79-4972-8717-19517d6034b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ea1b2837-ee79-4972-8717-19517d6034b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:43.533192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:43.534260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:43.535165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 155/305] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:43.536267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14916ffa-a481-4959-bf88-e57fa843e8fb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.536489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14916ffa-a481-4959-bf88-e57fa843e8fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.536763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14916ffa-a481-4959-bf88-e57fa843e8fb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.538123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 156/306] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:43.538815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14916ffa-a481-4959-bf88-e57fa843e8fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.550453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 151/307] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Jul 18 10:24:43 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:43.552533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.658697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14916ffa-a481-4959-bf88-e57fa843e8fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.673417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:43.690104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26de5f-ed75-4081-afbd-3352badfded8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.701883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92279a-136e-45d3-b9e0-a88867e42dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.702553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92279a-136e-45d3-b9e0-a88867e42dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.702831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92279a-136e-45d3-b9e0-a88867e42dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.703103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92279a-136e-45d3-b9e0-a88867e42dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.703951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92279a-136e-45d3-b9e0-a88867e42dc6 None admin] RBAC: Adding query filter params (role.id=21ae2370a18d4c9ebbd8a56f38d6b550, user.id=3fcfbe15957a4b71b896c63baf95775a, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:43.704263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92279a-136e-45d3-b9e0-a88867e42dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:43.705967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92279a-136e-45d3-b9e0-a88867e42dc6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:43.714486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 157/308] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/role_assignments?user.id=3fcfbe15957a4b71b896c63baf95775a&scope.system=all&role.id=21ae2370a18d4c9ebbd8a56f38d6b550 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:43.714486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f30ec-c1d0-411d-a071-f506362f7179 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.714486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f30ec-c1d0-411d-a071-f506362f7179 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.714486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f30ec-c1d0-411d-a071-f506362f7179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.735196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.735954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.737636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.737636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.738852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:43.739991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:43.741808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:43.742215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c47f30ec-c1d0-411d-a071-f506362f7179 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:43.743717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c47f30ec-c1d0-411d-a071-f506362f7179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:43.753599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 152/309] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:43.759624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.774196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:43.834998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 158/310] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:43.856269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f30ec-c1d0-411d-a071-f506362f7179 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.895630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49596af-c236-495d-84ca-d0e6dd0a6634 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.942132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:43.943581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:43.943869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:43.944150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] PATH_INFO: `/v3/users/6c9ea78355d943849807ec0496725dc7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:43.945574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] RBAC: Authorizing `identity:get_user(user_id=6c9ea78355d943849807ec0496725dc7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:43.947066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:43.948770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 153/311] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/users/6c9ea78355d943849807ec0496725dc7 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:43.954356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:43.975689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f30ec-c1d0-411d-a071-f506362f7179 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:43.985258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.014887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.015825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.016170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.016510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.029895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:44.038559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:44.039581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:44.045424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 159/312] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:44.047670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:44.068598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.088583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81836d83-fc91-4ef4-9de4-3244f8ae8b58 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.135868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.136821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.137162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.137445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] PATH_INFO: `/v3/system/users/6c9ea78355d943849807ec0496725dc7/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.139088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6c9ea78355d943849807ec0496725dc7, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:44.140588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:44.160657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195194-07e1-44df-97bd-e578fa49b301 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.196880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.197557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.197846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.198132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.198890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:44.199221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:44.200648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:44.210323 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 160/313] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:44.215747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:44.266729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 154/314] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Jul 18 10:24:43 2022] PUT /identity/v3/system/users/6c9ea78355d943849807ec0496725dc7/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:24:44.321144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.344881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedcae-c376-4aa7-9d91-bf308e599e46 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.387723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.389146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.389659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.390118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] PATH_INFO: `/v3/users/00e1244f84e84cbd876ef7ccdc632676` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.391894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] RBAC: Authorizing `identity:get_user(user_id=00e1244f84e84cbd876ef7ccdc632676)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:44.393782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:44.396805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 161/315] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/users/00e1244f84e84cbd876ef7ccdc632676 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:44.407423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:44.493294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.513090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d6c89-44ce-47c7-8e10-4a67bcafa15a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.548306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.548943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.549253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.549754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.621780 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:44.623252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:44.708437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.733123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 155/316] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:44.734976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:44.856432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5ab3d-985d-4768-a380-2d9b6c857918 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.892325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95e3f1-2ba6-4704-b5e5-4d4ad6f81e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.893516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95e3f1-2ba6-4704-b5e5-4d4ad6f81e98 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.894627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95e3f1-2ba6-4704-b5e5-4d4ad6f81e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.895148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95e3f1-2ba6-4704-b5e5-4d4ad6f81e98 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.896761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95e3f1-2ba6-4704-b5e5-4d4ad6f81e98 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:44.898487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95e3f1-2ba6-4704-b5e5-4d4ad6f81e98 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:44.904894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 162/317] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:44.906967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:45.022737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517117d7-1457-4e26-a5c7-00bc1a6cc9b9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:45.059600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b670dd-5e6c-429b-9c52-47c653c97563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:45.060551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b670dd-5e6c-429b-9c52-47c653c97563 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.061046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b670dd-5e6c-429b-9c52-47c653c97563 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.061520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b670dd-5e6c-429b-9c52-47c653c97563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.062534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b670dd-5e6c-429b-9c52-47c653c97563 None admin] RBAC: Adding query filter params (role.id=21ae2370a18d4c9ebbd8a56f38d6b550, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa, user.id=00e1244f84e84cbd876ef7ccdc632676) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:45.063027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b670dd-5e6c-429b-9c52-47c653c97563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:45.064662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b670dd-5e6c-429b-9c52-47c653c97563 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:45.071403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 156/318] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/role_assignments?user.id=00e1244f84e84cbd876ef7ccdc632676&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=21ae2370a18d4c9ebbd8a56f38d6b550 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:45.751843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749717-c77b-43cd-bf34-e7da75e40749 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.753163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749717-c77b-43cd-bf34-e7da75e40749 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.753657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749717-c77b-43cd-bf34-e7da75e40749 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.756514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 163/319] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:45.761667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56551d6d-85ad-4ad3-909b-7d2d3970b128 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.762123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56551d6d-85ad-4ad3-909b-7d2d3970b128 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.762292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56551d6d-85ad-4ad3-909b-7d2d3970b128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.779966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-56551d6d-85ad-4ad3-909b-7d2d3970b128 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:45.781200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-56551d6d-85ad-4ad3-909b-7d2d3970b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:45.829721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:45.857533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 157/320] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:45.880250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d749717-c77b-43cd-bf34-e7da75e40749 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:45.901552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:45.906044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6967e40-1bd4-43af-bdba-8b626db48265 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.906328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6967e40-1bd4-43af-bdba-8b626db48265 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.907089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6967e40-1bd4-43af-bdba-8b626db48265 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.908487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 158/321] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:45.914507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfdd91-6c36-4481-83e5-40b35e6c2aac None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.914797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfdd91-6c36-4481-83e5-40b35e6c2aac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.915075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfdd91-6c36-4481-83e5-40b35e6c2aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.933520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-34dfdd91-6c36-4481-83e5-40b35e6c2aac None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:45.934737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-34dfdd91-6c36-4481-83e5-40b35e6c2aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:46.012498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d749717-c77b-43cd-bf34-e7da75e40749 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.013743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 159/322] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:46.038913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfdd91-6c36-4481-83e5-40b35e6c2aac None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:46.060394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.061562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.062135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.062135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.063335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:46.064840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:46.067206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 164/323] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:46.072619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:46.150307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfdd91-6c36-4481-83e5-40b35e6c2aac None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.152634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:46.166031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:46.182961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec65c7-761e-4e70-9d7a-92f16d462069 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.187917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.188827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.189115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.189387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.203026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:46.212513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:46.215508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:46.216312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 160/324] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:46.219807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:46.225977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812531f7-2f7a-427c-a507-0fcb2cf9c867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.226647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812531f7-2f7a-427c-a507-0fcb2cf9c867 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.227012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812531f7-2f7a-427c-a507-0fcb2cf9c867 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.227310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812531f7-2f7a-427c-a507-0fcb2cf9c867 None admin] PATH_INFO: `/v3/groups` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.227985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812531f7-2f7a-427c-a507-0fcb2cf9c867 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:46.229564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812531f7-2f7a-427c-a507-0fcb2cf9c867 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:46.334543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a606d5-ba4e-4883-9c8a-29455671922b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.360859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 165/325] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Jul 18 10:24:46 2022] POST /identity/v3/groups => generated 226 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:24:46.360859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034e799-87db-47fd-9af1-a56c8363ee96 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.360859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034e799-87db-47fd-9af1-a56c8363ee96 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.360859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034e799-87db-47fd-9af1-a56c8363ee96 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.361238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 166/326] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:46.367675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b30aa-1e07-49ad-8927-f0931364aced None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.367968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b30aa-1e07-49ad-8927-f0931364aced None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.368252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b30aa-1e07-49ad-8927-f0931364aced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.380293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.381026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.381983 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.382272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.383055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:46.383480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:46.384981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:46.388062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-313b30aa-1e07-49ad-8927-f0931364aced None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:46.389356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-313b30aa-1e07-49ad-8927-f0931364aced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:46.395005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 161/327] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:46.400938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:46.433795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:46.477839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 167/328] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:46.501368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b30aa-1e07-49ad-8927-f0931364aced None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:46.520369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0088d-b095-4dfe-b0e8-432ed6c3d3f1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.561499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.562389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.562682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.562968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] PATH_INFO: `/v3/users/3fcfbe15957a4b71b896c63baf95775a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.564195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcfbe15957a4b71b896c63baf95775a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:46.565753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:46.568358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 162/329] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:46 2022] GET /identity/v3/users/3fcfbe15957a4b71b896c63baf95775a => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:46.574559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:46.595632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:46.633116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b30aa-1e07-49ad-8927-f0931364aced None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.680182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.681105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.681387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.681661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.695148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:46.695965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:46.702825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068f066-34d7-46d8-9bc8-74d99eeb1922 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:46.703809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:46.704682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:46.705648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 168/330] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:46.713810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:46.761444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.762416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.763000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.763148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] PATH_INFO: `/v3/system/users/3fcfbe15957a4b71b896c63baf95775a/roles/21ae2370a18d4c9ebbd8a56f38d6b550` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.765600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3fcfbe15957a4b71b896c63baf95775a, role_id=21ae2370a18d4c9ebbd8a56f38d6b550)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:46.767432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:46.844911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e2be2-8b8c-4447-a41c-fb6d43685244 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:46.866377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 163/331] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Jul 18 10:24:46 2022] PUT /identity/v3/system/users/3fcfbe15957a4b71b896c63baf95775a/roles/21ae2370a18d4c9ebbd8a56f38d6b550 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:24:46.884691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:46.885706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:46.886158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:46.886584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:46.887697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:46.888195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:46.889857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:46.891830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:46.902819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 169/332] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:46.908712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:47.036543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafe1b2-0811-4638-933c-bd7a7ee05c40 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:47.077024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa97cf-e919-44bd-828d-897102daadfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:47.077884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa97cf-e919-44bd-828d-897102daadfa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:47.078214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa97cf-e919-44bd-828d-897102daadfa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:47.078445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa97cf-e919-44bd-828d-897102daadfa None admin] PATH_INFO: `/v3/users/6c9ea78355d943849807ec0496725dc7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:47.079648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa97cf-e919-44bd-828d-897102daadfa None admin] RBAC: Authorizing `identity:get_user(user_id=6c9ea78355d943849807ec0496725dc7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:47.081191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa97cf-e919-44bd-828d-897102daadfa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:47.083796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 164/333] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:46 2022] GET /identity/v3/users/6c9ea78355d943849807ec0496725dc7 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:47.089722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:47.241002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee00c0-8c69-440e-a1d9-bb2b3978ff55 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:47.286073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d732-f7f6-4ed5-bac5-567b6f2b4b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:47.287101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d732-f7f6-4ed5-bac5-567b6f2b4b7a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:47.287627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d732-f7f6-4ed5-bac5-567b6f2b4b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:47.288099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d732-f7f6-4ed5-bac5-567b6f2b4b7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:47.289186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d732-f7f6-4ed5-bac5-567b6f2b4b7a None admin] RBAC: Adding query filter params (role.id=9cc91942b0de4b1bb86179386ef67e09, user.id=6c9ea78355d943849807ec0496725dc7, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:47.289700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d732-f7f6-4ed5-bac5-567b6f2b4b7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:47.291358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d732-f7f6-4ed5-bac5-567b6f2b4b7a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:47.309674 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 170/334] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 18 10:24:47 2022] GET /identity/v3/role_assignments?user.id=6c9ea78355d943849807ec0496725dc7&scope.system=all&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:48.782520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65937741-0c22-4031-a4e4-4d3d3e990dd1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:48.783031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65937741-0c22-4031-a4e4-4d3d3e990dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:48.783105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65937741-0c22-4031-a4e4-4d3d3e990dd1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:48.785196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 165/335] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:48.791087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f1a1-72a8-4cfc-8791-73038768bf8c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:48.791551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f1a1-72a8-4cfc-8791-73038768bf8c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:48.791733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f1a1-72a8-4cfc-8791-73038768bf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:48.830178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0101f1a1-72a8-4cfc-8791-73038768bf8c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:48.832048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0101f1a1-72a8-4cfc-8791-73038768bf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:48.905159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:48.925688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 171/336] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:48.952905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65937741-0c22-4031-a4e4-4d3d3e990dd1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:48.995171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:49.092977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65937741-0c22-4031-a4e4-4d3d3e990dd1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:49.138532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:49.139369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:49.139832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:49.140154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:49.155513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:49.164756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:49.165751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:49.167023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 166/337] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:49.172610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101f1a1-72a8-4cfc-8791-73038768bf8c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:49.314209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101f1a1-72a8-4cfc-8791-73038768bf8c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:49.364033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcdfcc-878c-4cb0-91a2-2d9a96b4e668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:49.365083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcdfcc-878c-4cb0-91a2-2d9a96b4e668 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:49.365565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcdfcc-878c-4cb0-91a2-2d9a96b4e668 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:49.366060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcdfcc-878c-4cb0-91a2-2d9a96b4e668 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:49.367027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcdfcc-878c-4cb0-91a2-2d9a96b4e668 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:49.367532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcdfcc-878c-4cb0-91a2-2d9a96b4e668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:49.369158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcdfcc-878c-4cb0-91a2-2d9a96b4e668 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:49.382018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 172/338] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:49.387395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:49.515857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:49.532035 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ceacd-4add-4693-9c13-628bf87c74aa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:49.577633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd81caa-1890-40fa-8883-a71c25a75b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:49.578315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd81caa-1890-40fa-8883-a71c25a75b0c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:49.578607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd81caa-1890-40fa-8883-a71c25a75b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:49.578889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd81caa-1890-40fa-8883-a71c25a75b0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:49.677413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd81caa-1890-40fa-8883-a71c25a75b0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:49.679223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd81caa-1890-40fa-8883-a71c25a75b0c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:49.715691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d004d4-43a6-4e83-a934-1ff5e0f5cbfc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:49.716011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d004d4-43a6-4e83-a934-1ff5e0f5cbfc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:49.716251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d004d4-43a6-4e83-a934-1ff5e0f5cbfc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:49.717915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 173/339] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:49.723679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa8fd6-cc0a-4f6b-893f-bbf4078881b7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:49.724251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa8fd6-cc0a-4f6b-893f-bbf4078881b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:49.724722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa8fd6-cc0a-4f6b-893f-bbf4078881b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:49.754357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-78aa8fd6-cc0a-4f6b-893f-bbf4078881b7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:49.758079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-78aa8fd6-cc0a-4f6b-893f-bbf4078881b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:49.836896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:49.855073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:49.868013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 167/340] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:49.872216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354827f4-612e-44ed-99db-6f7964f3c631 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:49.872499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354827f4-612e-44ed-99db-6f7964f3c631 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:49.872800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354827f4-612e-44ed-99db-6f7964f3c631 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:49.874845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 168/341] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:49.875659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354827f4-612e-44ed-99db-6f7964f3c631 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:49.920070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 174/342] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:24:49.921322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42726e-9161-444b-a5e9-c029be37d650 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:49.921617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42726e-9161-444b-a5e9-c029be37d650 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:49.921889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42726e-9161-444b-a5e9-c029be37d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:49.961165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-af42726e-9161-444b-a5e9-c029be37d650 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:49.962410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af42726e-9161-444b-a5e9-c029be37d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:50.001975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354827f4-612e-44ed-99db-6f7964f3c631 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.057387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.058257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.058542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.058820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.061838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:50.064287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:50.067499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 169/343] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:49 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:50.068324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.081615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 175/344] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:50.084264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8902a-ff0d-4370-ad7f-d0357a9b646c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.085097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8902a-ff0d-4370-ad7f-d0357a9b646c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.085579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8902a-ff0d-4370-ad7f-d0357a9b646c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.088186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 176/345] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:50.089199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8902a-ff0d-4370-ad7f-d0357a9b646c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.147337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:50.205863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17954df1-9ffe-4917-ab58-c0a53277729a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.221615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8902a-ff0d-4370-ad7f-d0357a9b646c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.232618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:50.246271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c731f-c6b4-43ac-ad9a-76ee92ae90c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.247119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c731f-c6b4-43ac-ad9a-76ee92ae90c0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.247407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c731f-c6b4-43ac-ad9a-76ee92ae90c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.247686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c731f-c6b4-43ac-ad9a-76ee92ae90c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.259929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c731f-c6b4-43ac-ad9a-76ee92ae90c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-dd9c731f-c6b4-43ac-ad9a-76ee92ae90c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:50.268226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:50.270852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 170/346] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:50.271878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d26f84-1b62-4e5b-9e67-51bb8a490d3b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.272577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d26f84-1b62-4e5b-9e67-51bb8a490d3b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.272577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d26f84-1b62-4e5b-9e67-51bb8a490d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.274658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.274658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.274658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.275357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.287788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:50.289664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:50.293029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 177/347] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:50.293624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-58d26f84-1b62-4e5b-9e67-51bb8a490d3b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:50.294323 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.294828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-58d26f84-1b62-4e5b-9e67-51bb8a490d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:50.383719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 171/348] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:50.384591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d26f84-1b62-4e5b-9e67-51bb8a490d3b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.439075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8d498-b357-4a9b-a979-c311f7b4ea2d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.464966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:50.486163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.486900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.487295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.488196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.507148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:50.515595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:50.516535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:50.517836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 178/349] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:50.518796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.528359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d26f84-1b62-4e5b-9e67-51bb8a490d3b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.560978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:50.573763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.574450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.574736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.575069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.575839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:50.576145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:50.577556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:50.589756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 172/350] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:50.592365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.662938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817adee7-5646-44ef-8d96-25b02fb6298c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.709484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.710143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.710426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.710698 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.711534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=8efd660d626b4141b5c9be1824e7e416) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:50.711844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:50.713352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:50.721131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 179/351] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:50.722597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.726978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25af89-6fb4-4acc-a2c8-e049b41e7b66 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.761259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:50.773590 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.774411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.774797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.775088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.789550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:50.798910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:50.800222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 173/352] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:50.801545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.889448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:50.906299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093847-503e-455c-beb2-df9659f07d1b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.929507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ce956-db51-4411-9459-00bb7c609cf5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.962734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.963419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.963700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.963979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.964729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:50.965089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:50.966531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:50.973212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13a666-4572-4620-9b96-da65b0ba0ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.974108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13a666-4572-4620-9b96-da65b0ba0ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.974390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13a666-4572-4620-9b96-da65b0ba0ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.974675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13a666-4572-4620-9b96-da65b0ba0ea4 None admin] PATH_INFO: `/v3/users/3fcfbe15957a4b71b896c63baf95775a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.976036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13a666-4572-4620-9b96-da65b0ba0ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcfbe15957a4b71b896c63baf95775a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:50.977730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13a666-4572-4620-9b96-da65b0ba0ea4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:50.978984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 180/353] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:50.979944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 174/354] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/users/3fcfbe15957a4b71b896c63baf95775a => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:50.980736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.982039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed8332-1f0b-4a6f-816d-4d86a4aee74c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.982325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed8332-1f0b-4a6f-816d-4d86a4aee74c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.982604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed8332-1f0b-4a6f-816d-4d86a4aee74c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.984170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 175/355] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:50.985381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed8332-1f0b-4a6f-816d-4d86a4aee74c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.034735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.123030 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da895f-2a4f-473b-bfa6-dc425c49a4c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.125087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed8332-1f0b-4a6f-816d-4d86a4aee74c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.170080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f991d-c1d5-4181-a922-d777bc3ab2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.171093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f991d-c1d5-4181-a922-d777bc3ab2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.171206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.171648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f991d-c1d5-4181-a922-d777bc3ab2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.171787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.172077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.172321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f991d-c1d5-4181-a922-d777bc3ab2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.172440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.174040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f991d-c1d5-4181-a922-d777bc3ab2f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:51.174492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f991d-c1d5-4181-a922-d777bc3ab2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.176855 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f991d-c1d5-4181-a922-d777bc3ab2f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.188223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 181/356] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:51.190093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e6df6-a63d-4451-88fc-fd25abb42e6a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.191163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e6df6-a63d-4451-88fc-fd25abb42e6a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.191832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e6df6-a63d-4451-88fc-fd25abb42e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.214468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-981e6df6-a63d-4451-88fc-fd25abb42e6a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:51.216289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-981e6df6-a63d-4451-88fc-fd25abb42e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:51.236049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.274708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.274708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.311219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 182/357] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:51.312285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e6df6-a63d-4451-88fc-fd25abb42e6a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.321692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.417282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 176/358] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:51.418350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.455125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e6df6-a63d-4451-88fc-fd25abb42e6a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.501632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.502757 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.503267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.503830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.505811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] RBAC: Adding query filter params (role.id=21ae2370a18d4c9ebbd8a56f38d6b550, user.id=3fcfbe15957a4b71b896c63baf95775a, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:51.505811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.506923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.528626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 183/359] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/role_assignments?user.id=3fcfbe15957a4b71b896c63baf95775a&scope.system=all&role.id=21ae2370a18d4c9ebbd8a56f38d6b550 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:51.529331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.546964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.552736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f19e6-8150-404b-8189-2605dffba418 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.594111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1c911-84ec-41c0-8282-18b5f3224b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.594688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1c911-84ec-41c0-8282-18b5f3224b19 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.594993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1c911-84ec-41c0-8282-18b5f3224b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.595244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1c911-84ec-41c0-8282-18b5f3224b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.599950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.677741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192947b9-b57c-4a90-960c-d194c567b787 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.681071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1c911-84ec-41c0-8282-18b5f3224b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.682577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1c911-84ec-41c0-8282-18b5f3224b19 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.730398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.731239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.731525 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.731805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.746246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:51.754416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:51.755330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:51.756119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:51.756733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 184/360] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:51.757358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.817065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 177/361] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:51.817065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa63498-c88a-4286-9e12-41d9b4ed4c02 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.817065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa63498-c88a-4286-9e12-41d9b4ed4c02 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.817838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa63498-c88a-4286-9e12-41d9b4ed4c02 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.820578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 178/362] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:51.821405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa63498-c88a-4286-9e12-41d9b4ed4c02 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.849443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.867265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.896049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fad364-3980-4b58-a6b1-661adb1c1218 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.944125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.945018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.945301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.945573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.946804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.948426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.950454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 185/363] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:51.951386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.954107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa63498-c88a-4286-9e12-41d9b4ed4c02 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.003530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83aaeb-e80b-4f27-92d2-338a637722a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.004684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83aaeb-e80b-4f27-92d2-338a637722a5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.005043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83aaeb-e80b-4f27-92d2-338a637722a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.005376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83aaeb-e80b-4f27-92d2-338a637722a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.006191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83aaeb-e80b-4f27-92d2-338a637722a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:52.006475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83aaeb-e80b-4f27-92d2-338a637722a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.007865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83aaeb-e80b-4f27-92d2-338a637722a5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.017224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 179/364] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:52.018484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95177ab-58af-4a17-8328-11a84ababc2e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.018875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95177ab-58af-4a17-8328-11a84ababc2e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.019167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95177ab-58af-4a17-8328-11a84ababc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.037657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a95177ab-58af-4a17-8328-11a84ababc2e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:52.038845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a95177ab-58af-4a17-8328-11a84ababc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:52.084196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a26c3-03e6-4bd5-901e-790fc6755f41 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.124445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.125279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.125564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.125848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.127307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.128822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 180/365] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:52.129040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.130035 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95177ab-58af-4a17-8328-11a84ababc2e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.130992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 186/366] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:52.131719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.134753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:52.168860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:52.267759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95177ab-58af-4a17-8328-11a84ababc2e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.272857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1412-b1ce-4c7e-88d0-456e26621ac3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.313024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.313907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.314191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.314462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.316176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.318088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.318481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.319167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.319452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.319769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.320029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 181/367] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:52.320822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.419369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.420839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.423994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:52.473171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:52.482151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2ad53-b167-4cd0-98a9-04f00218d948 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.541924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.542796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.543841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.543841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.547815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.549142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.552589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 182/368] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:52.552839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.567802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 187/369] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:52.568753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.709467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efad9ca-7a42-40e1-af14-33c38a301104 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.711352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:52.713367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc6635-8888-452a-aed7-bdd728a90659 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.755540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.756528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.757031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.758449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.759292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=8efd660d626b4141b5c9be1824e7e416) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:52.759579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.761218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.762473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.763474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.763805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.764087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.767927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 188/370] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=4f582d9851794ff894a232e045525ec4 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:52.768876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.781939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:52.791492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:52.792439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:52.793315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:52.795972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 183/371] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:52.797202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.838190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:52.925541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab7d5-cfc8-42a8-b340-220a8abee03e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.938041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673cba0-1136-4a9f-9269-85f9548c02b8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:52.979337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.979913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.980192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.980464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.981340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:52.981651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.983129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.986772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:52.989926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:52.990397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:52.990866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:52.991977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 189/372] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:52.992456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:52.993186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:52.994390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:52.996954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 184/373] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:53.008541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:53.042028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:53.136744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:53.139231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a821e8-8d99-4aca-a636-e5b112e32f72 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:53.151008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec2db3-109f-46fc-a7b2-7675a1344fe2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:53.185768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:53.186593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.187194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.187431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.188326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:53.188724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:53.191153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:53.202474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 190/374] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:53.203470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:53.204715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.205199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.206125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.207526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:53.208964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:53.209607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:53.211718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 185/375] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:53 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:53.217657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:53.326884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:53.347525 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4e5e0-c6bd-4be1-af77-23217f9e4219 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:53.350841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e789-7c99-4243-9a5e-b9203293a25b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:53.393154 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:53.393831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.394142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.394428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.397580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6017e-b27d-4c8c-91ae-149345acc034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:53.398758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6017e-b27d-4c8c-91ae-149345acc034 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.400194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6017e-b27d-4c8c-91ae-149345acc034 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.400635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6017e-b27d-4c8c-91ae-149345acc034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.401698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6017e-b27d-4c8c-91ae-149345acc034 None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:53.402167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6017e-b27d-4c8c-91ae-149345acc034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:53.403790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6017e-b27d-4c8c-91ae-149345acc034 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:53.412310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 186/376] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:24:53 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=4f582d9851794ff894a232e045525ec4 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:53.414577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c066-605e-4f48-8167-2229074b62bd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.415158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c066-605e-4f48-8167-2229074b62bd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.415587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c066-605e-4f48-8167-2229074b62bd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.418377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 187/377] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:53.424275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1449-1402-4bd0-a924-86287f211388 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.425372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1449-1402-4bd0-a924-86287f211388 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.425818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1449-1402-4bd0-a924-86287f211388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.449971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ba8f1449-1402-4bd0-a924-86287f211388 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:53.451461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba8f1449-1402-4bd0-a924-86287f211388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:53.489188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:53.493381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:53.494981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:53.552517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 188/378] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:53.577382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1449-1402-4bd0-a924-86287f211388 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:53.638835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:53.660828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 191/379] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:24:53.666475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:53.710417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1449-1402-4bd0-a924-86287f211388 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:53.749505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:53.750699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.751125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.751542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.764481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:53.774895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:53.775995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:53.777010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:53.779583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 189/380] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:53.786976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:53.793194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24797e17-3308-418b-9c38-8a6ad86abda1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:53.829477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:53.858439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:53.859056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.859347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.859622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] PATH_INFO: `/v3/projects/233b60ec452b417c902a9645e05969db` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.860883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] RBAC: Authorizing `identity:get_project(project_id=233b60ec452b417c902a9645e05969db)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:53.862910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:53.864983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 192/381] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:53 2022] GET /identity/v3/projects/233b60ec452b417c902a9645e05969db => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:53.870177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:53.923107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9156fe-5c9e-4ea5-9b97-528d9d3058f3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:53.971490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:53.973604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:53.974259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:53.975257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:53.975873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:53.977557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:53.978760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:53.982493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:53.998648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 190/382] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:54.017934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:54.071642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e896c86-0a6e-4b45-b8ca-e9f332581955 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:54.146286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:54.147079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:54.147360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:54.147636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] PATH_INFO: `/v3/groups/5950e7ed89754f02890a194ed5c2dca7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:54.161677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] RBAC: Authorizing `identity:get_group(group_id=5950e7ed89754f02890a194ed5c2dca7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:54.163195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:54.164960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 193/383] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:53 2022] GET /identity/v3/groups/5950e7ed89754f02890a194ed5c2dca7 => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:54.165205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:54.177687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:54.201706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0ebc1-ea32-42ed-8942-c6a5a85fb982 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:54.271535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:54.272417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:54.272705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:54.273008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:54.274389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:54.275979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:54.278695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 191/384] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:54 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:54.289866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:54.395773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc509c4-f3db-42d7-9271-f9a8e8858208 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:54.449149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:54.473062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:54.473803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:54.474091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:54.474366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:54.475363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] RBAC: Adding query filter params (group.id=5950e7ed89754f02890a194ed5c2dca7, role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=233b60ec452b417c902a9645e05969db) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:54.475679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:54.477179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:54.489365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 194/385] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:24:54 2022] GET /identity/v3/role_assignments?group.id=5950e7ed89754f02890a194ed5c2dca7&scope.project.id=233b60ec452b417c902a9645e05969db&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 260 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:54.492007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad1026-d07e-46f0-ae3b-0514348b5fcf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:54.532558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665f630-27b2-47f1-9c2e-fe766e990c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:54.533476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665f630-27b2-47f1-9c2e-fe766e990c72 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:54.534315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665f630-27b2-47f1-9c2e-fe766e990c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:54.534647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665f630-27b2-47f1-9c2e-fe766e990c72 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:54.536021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665f630-27b2-47f1-9c2e-fe766e990c72 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:54.537528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665f630-27b2-47f1-9c2e-fe766e990c72 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:54.539609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 192/386] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:54 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:54.545355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:54.683161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621612d-e46b-4dac-b596-f46b435faa6c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:54.724642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcc0c9-2d5b-4229-aa0e-8a15a912f695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:54.725511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcc0c9-2d5b-4229-aa0e-8a15a912f695 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:54.725805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcc0c9-2d5b-4229-aa0e-8a15a912f695 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:54.726084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcc0c9-2d5b-4229-aa0e-8a15a912f695 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:54.728141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcc0c9-2d5b-4229-aa0e-8a15a912f695 None admin] RBAC: Authorizing `identity:create_grant(project_id=8efd660d626b4141b5c9be1824e7e416, group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:54.729711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcc0c9-2d5b-4229-aa0e-8a15a912f695 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:54.815741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 195/387] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 18 10:24:54 2022] PUT /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:55.499867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9ed8e-10a6-4c29-84e6-3defd31c2124 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:55.500919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9ed8e-10a6-4c29-84e6-3defd31c2124 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:55.501431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9ed8e-10a6-4c29-84e6-3defd31c2124 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:55.504333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 193/388] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:55.509993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c91316-9e40-4490-bcac-81f142d8672f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:55.510852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c91316-9e40-4490-bcac-81f142d8672f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:55.511430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c91316-9e40-4490-bcac-81f142d8672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:55.530268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6c91316-9e40-4490-bcac-81f142d8672f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:55.531874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6c91316-9e40-4490-bcac-81f142d8672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:55.599528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:55.625200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 196/389] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:55.650917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9ed8e-10a6-4c29-84e6-3defd31c2124 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:55.665359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:55.797648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9ed8e-10a6-4c29-84e6-3defd31c2124 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:55.821425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410dc24-01e2-4f63-b901-0a1847afd06a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:55.821731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410dc24-01e2-4f63-b901-0a1847afd06a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:55.822017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410dc24-01e2-4f63-b901-0a1847afd06a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:55.823772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 197/390] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:55.829793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b041f6d-9042-471d-af0b-af6e5a7d2c2d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:55.830088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b041f6d-9042-471d-af0b-af6e5a7d2c2d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:55.831179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b041f6d-9042-471d-af0b-af6e5a7d2c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:55.855249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:55.856077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:55.856420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:55.856698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:55.863610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0b041f6d-9042-471d-af0b-af6e5a7d2c2d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:55.865450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b041f6d-9042-471d-af0b-af6e5a7d2c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:55.886551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:55.899905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:55.900910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:55.901792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 194/391] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:55.905806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.025641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 198/392] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:56.049284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.054478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b041f6d-9042-471d-af0b-af6e5a7d2c2d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.090982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb46de-bcb0-48a3-9e48-7f6534ddfcf6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.165133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b041f6d-9042-471d-af0b-af6e5a7d2c2d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.175309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.181480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.182205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.182501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.189422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.189422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:56.189422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.189422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.203508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.204330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.204617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.205208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.210909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 195/393] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:56.216259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.217493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:56.228583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:56.229548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:56.230339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:56.231399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 199/394] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:56.237516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.348573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246991b1-c965-4a45-9a25-d7021bfcfb05 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.354153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567070d5-41e8-499b-adac-90b5a08828a3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.382898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.388707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04b4ce-335a-4ccd-95b6-547ecf4ef275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.389686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04b4ce-335a-4ccd-95b6-547ecf4ef275 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.390060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04b4ce-335a-4ccd-95b6-547ecf4ef275 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.390373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04b4ce-335a-4ccd-95b6-547ecf4ef275 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.391724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04b4ce-335a-4ccd-95b6-547ecf4ef275 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.393706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04b4ce-335a-4ccd-95b6-547ecf4ef275 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.396094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 196/395] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:56.400862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce685a-f2d2-4254-8f58-a2154a445fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.400996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397182b-8537-44f1-b797-d0b5d0970223 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.400996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397182b-8537-44f1-b797-d0b5d0970223 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.401150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce685a-f2d2-4254-8f58-a2154a445fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.401244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397182b-8537-44f1-b797-d0b5d0970223 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.401335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce685a-f2d2-4254-8f58-a2154a445fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.401335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce685a-f2d2-4254-8f58-a2154a445fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.401335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce685a-f2d2-4254-8f58-a2154a445fb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:56.401541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 197/396] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:56.401617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce685a-f2d2-4254-8f58-a2154a445fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.401961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce685a-f2d2-4254-8f58-a2154a445fb7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.404050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397182b-8537-44f1-b797-d0b5d0970223 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.413961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 200/397] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:56.415383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ad58-cbd8-4739-a1a3-657952e37ee1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.415809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ad58-cbd8-4739-a1a3-657952e37ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.416200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ad58-cbd8-4739-a1a3-657952e37ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.442181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9418ad58-cbd8-4739-a1a3-657952e37ee1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:56.443599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9418ad58-cbd8-4739-a1a3-657952e37ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:56.477352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.534298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 201/398] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:56.541137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418ad58-cbd8-4739-a1a3-657952e37ee1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.578665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397182b-8537-44f1-b797-d0b5d0970223 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.626711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.627875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.628164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.628738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.630127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.631702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.634041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 198/399] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:56.634905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.669684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418ad58-cbd8-4739-a1a3-657952e37ee1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.707304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.708169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.708452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.708729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.710142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.711249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.712833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.714769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 202/400] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:56.715832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.746117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.751892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2c27c-3869-42ab-8e11-8af4fd72f8a5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.789957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.791306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.791487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.791881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.815954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:56.826542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:56.827772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:56.828751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 199/401] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:56.829539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.831765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3afa47-f8f2-4195-b001-aae061e8fa55 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.867698 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b216b5-463f-423b-86e8-3632612caa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.868596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b216b5-463f-423b-86e8-3632612caa8a None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.868939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b216b5-463f-423b-86e8-3632612caa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.869207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b216b5-463f-423b-86e8-3632612caa8a None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.871180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b216b5-463f-423b-86e8-3632612caa8a None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6581cd5cb54c7aacb51e90736e34aa, group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.872639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b216b5-463f-423b-86e8-3632612caa8a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.940413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0b73c-776c-48e2-8813-1194c6230e81 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.953014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 203/402] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 18 10:24:56 2022] PUT /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:56.953014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0f022-ec6f-4e0a-8681-ad0ca0af0b79 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.953014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0f022-ec6f-4e0a-8681-ad0ca0af0b79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.953014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0f022-ec6f-4e0a-8681-ad0ca0af0b79 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.953014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 204/403] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:56.953014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0f022-ec6f-4e0a-8681-ad0ca0af0b79 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.987366 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.000077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25462bcb-acd5-460b-be67-41b41afbf80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.002704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462bcb-acd5-460b-be67-41b41afbf80d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.002704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462bcb-acd5-460b-be67-41b41afbf80d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.002704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462bcb-acd5-460b-be67-41b41afbf80d None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.003930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25462bcb-acd5-460b-be67-41b41afbf80d None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.005629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25462bcb-acd5-460b-be67-41b41afbf80d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.008614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 200/404] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:57.010862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f1d2c-ca77-4a49-b2d0-7053fd666ab2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.011809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f1d2c-ca77-4a49-b2d0-7053fd666ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.012680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f1d2c-ca77-4a49-b2d0-7053fd666ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.038109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-043f1d2c-ca77-4a49-b2d0-7053fd666ab2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:57.040408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-043f1d2c-ca77-4a49-b2d0-7053fd666ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.076080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.091072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0f022-ec6f-4e0a-8681-ad0ca0af0b79 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.126947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 201/405] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:57.128348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f1d2c-ca77-4a49-b2d0-7053fd666ab2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.133320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.134996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.135348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.135695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.136587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:57.136992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.139769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.150705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 205/406] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:57.154298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.247941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f1d2c-ca77-4a49-b2d0-7053fd666ab2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.277003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.286896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c3d6-74bb-400d-a404-a1d548ac65bf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.287206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.288110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.288444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.288725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.291892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] RBAC: Authorizing `identity:create_grant(project_id=8efd660d626b4141b5c9be1824e7e416, group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.293470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.328145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.329277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.329815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.330863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.343585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:57.352249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:57.353420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:57.354268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:57.358601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 206/407] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:57.359335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.373154 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.436844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 202/408] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 18 10:24:57 2022] PUT /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/4f582d9851794ff894a232e045525ec4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:57.436844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.510541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc04960-8369-481c-80bc-692e3037f801 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.571072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8cd4-bc12-457d-8b04-b2e16fcf38b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.572000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8cd4-bc12-457d-8b04-b2e16fcf38b5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.572296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8cd4-bc12-457d-8b04-b2e16fcf38b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.572579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8cd4-bc12-457d-8b04-b2e16fcf38b5 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.573913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac8cd4-bc12-457d-8b04-b2e16fcf38b5 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.575708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac8cd4-bc12-457d-8b04-b2e16fcf38b5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.577603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 207/409] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:57.578843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023ab7-fee3-4cad-8795-67914bb1b5c1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.579208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023ab7-fee3-4cad-8795-67914bb1b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.579495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023ab7-fee3-4cad-8795-67914bb1b5c1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.581100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 208/410] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:57.584021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41023ab7-fee3-4cad-8795-67914bb1b5c1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.617255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fd21b-26c7-4840-b2e6-02ea072f4762 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.657317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d352df5-8da0-42e6-9666-672f6f3cb591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.657809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d352df5-8da0-42e6-9666-672f6f3cb591 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.658091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d352df5-8da0-42e6-9666-672f6f3cb591 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.658900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d352df5-8da0-42e6-9666-672f6f3cb591 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.661605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d352df5-8da0-42e6-9666-672f6f3cb591 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:57.661605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d352df5-8da0-42e6-9666-672f6f3cb591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.661800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d352df5-8da0-42e6-9666-672f6f3cb591 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.671634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 203/411] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:57.673665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382622d2-16c5-4cb2-8511-cf13edf8af8e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.673776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382622d2-16c5-4cb2-8511-cf13edf8af8e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.674204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382622d2-16c5-4cb2-8511-cf13edf8af8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.691799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-382622d2-16c5-4cb2-8511-cf13edf8af8e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:57.692675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.693820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-382622d2-16c5-4cb2-8511-cf13edf8af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.718717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41023ab7-fee3-4cad-8795-67914bb1b5c1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.757884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.760632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.761732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.762173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.762552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.763920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.765764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.768747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 209/412] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:57.773940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.789787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 204/413] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:57.796814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382622d2-16c5-4cb2-8511-cf13edf8af8e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.913774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a8484-64d3-4098-b971-4f1acbc76938 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.949814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382622d2-16c5-4cb2-8511-cf13edf8af8e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.954637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.956067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.956353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.956631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] PATH_INFO: `/v3/groups/5950e7ed89754f02890a194ed5c2dca7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.957913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] RBAC: Authorizing `identity:get_group(group_id=5950e7ed89754f02890a194ed5c2dca7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.959389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.961419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 210/414] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/groups/5950e7ed89754f02890a194ed5c2dca7 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:57.962318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.981299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.005057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.006362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.006840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.007307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.010258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6581cd5cb54c7aacb51e90736e34aa, group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.012383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.075692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f7267-6d99-4271-aa6b-4b31519506f7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.113093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 205/415] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 18 10:24:57 2022] PUT /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72/roles/4f582d9851794ff894a232e045525ec4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:58.115121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:58.116334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.117697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.118186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.118683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.135176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:58.147401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:58.148451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:24:58.154798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 211/416] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:58.157109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:58.180014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.249473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.278591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409b42-c28e-44f3-9464-ddab8532b0d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.281035 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821a3ef-0aaf-4f25-b367-ad6ffe3bac35 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.322651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.323585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.323975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.324403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.325307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:58.328639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.330366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.336839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.338027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.342162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.342663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] PATH_INFO: `/v3/projects/233b60ec452b417c902a9645e05969db` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.344363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] RBAC: Authorizing `identity:get_project(project_id=233b60ec452b417c902a9645e05969db)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.346397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.353648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 206/417] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:58 2022] GET /identity/v3/projects/233b60ec452b417c902a9645e05969db => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:58.354105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 212/418] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:58.357181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:58.361895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:58.498209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901df3-e04d-48cc-82b2-92da8f11b148 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.498499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7561970-74d8-43ba-a3c9-aa84f99af4ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.507097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.537418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.537998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.538382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.538672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.538933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.539945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.540333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.540714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] PATH_INFO: `/v3/projects/233b60ec452b417c902a9645e05969db/groups/5950e7ed89754f02890a194ed5c2dca7/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.542666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.543702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] RBAC: Authorizing `identity:create_grant(project_id=233b60ec452b417c902a9645e05969db, group_id=5950e7ed89754f02890a194ed5c2dca7, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.545331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.616448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.618106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.673412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 207/419] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 18 10:24:58 2022] PUT /identity/v3/projects/233b60ec452b417c902a9645e05969db/groups/5950e7ed89754f02890a194ed5c2dca7/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:58.734977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 213/420] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:24:58.740664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:58.902953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a683dce-92ae-436c-9d88-00dcc96f3b6f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.950134 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7826b8-35c3-481c-b105-3ff450a3edba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.951436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7826b8-35c3-481c-b105-3ff450a3edba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.951944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7826b8-35c3-481c-b105-3ff450a3edba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.952437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7826b8-35c3-481c-b105-3ff450a3edba None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.957685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7826b8-35c3-481c-b105-3ff450a3edba None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.959480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7826b8-35c3-481c-b105-3ff450a3edba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.961871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 208/421] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:58 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:58.967675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:59.007652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:59.099692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755b935-4963-4fed-9a1e-783d354464ff None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.099987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755b935-4963-4fed-9a1e-783d354464ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.100174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755b935-4963-4fed-9a1e-783d354464ff None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:59.101814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 209/422] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:59.102554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9d01b-7cd3-4ac3-9d3a-897c3b78414e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:59.108409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab2f41-c481-44af-9ffc-0ce603f2c851 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.108803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab2f41-c481-44af-9ffc-0ce603f2c851 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.109088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab2f41-c481-44af-9ffc-0ce603f2c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:59.137837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-39ab2f41-c481-44af-9ffc-0ce603f2c851 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:59.139049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-39ab2f41-c481-44af-9ffc-0ce603f2c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:59.147096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:59.147981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.148270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.148547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:59.149835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:59.151900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:59.153887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 214/423] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:24:58 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:59.160629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:59.210192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:59.302663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 210/424] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:24:59.320359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d475e-8163-4bed-bd3c-6709a6df1ff9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:59.325778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:59.348903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab2f41-c481-44af-9ffc-0ce603f2c851 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:59.425684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:59.426421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.426764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.427075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:59.428117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=8efd660d626b4141b5c9be1824e7e416) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:59.428466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:59.430464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:59.444702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 215/425] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:24:59 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 611 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:59.482394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab2f41-c481-44af-9ffc-0ce603f2c851 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:59.538043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:59.538864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.539148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.539423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:59.561299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:59.570038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:59.571029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:24:59.572023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 211/426] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:59.577470 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:59.699422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef195-5d5c-4717-a50c-a8d74651b695 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:59.744813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df2d6a-be29-49c7-9d73-4581e4cddb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:59.745515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2d6a-be29-49c7-9d73-4581e4cddb92 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.745794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2d6a-be29-49c7-9d73-4581e4cddb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.746072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2d6a-be29-49c7-9d73-4581e4cddb92 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:59.746843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df2d6a-be29-49c7-9d73-4581e4cddb92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:59.747178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df2d6a-be29-49c7-9d73-4581e4cddb92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:59.748591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df2d6a-be29-49c7-9d73-4581e4cddb92 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:59.758845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 216/427] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:24:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:59.764197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:59.805638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:59.893977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52fe26-9baf-43fc-b1d1-cef322d50dbc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:59.938350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:59.939066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.939955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.940253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.022408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:00.023808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:00.053249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:00.106642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6ef5-e344-4169-aceb-89f6a61a2b7b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.106991 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6ef5-e344-4169-aceb-89f6a61a2b7b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.107282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6ef5-e344-4169-aceb-89f6a61a2b7b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.110350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 217/428] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:00.116512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e8cd-4f81-4899-8a95-b6c17e1a9a10 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.116825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e8cd-4f81-4899-8a95-b6c17e1a9a10 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.117114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e8cd-4f81-4899-8a95-b6c17e1a9a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.136810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-70a7e8cd-4f81-4899-8a95-b6c17e1a9a10 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:00.138119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-70a7e8cd-4f81-4899-8a95-b6c17e1a9a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:00.160199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 212/429] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:00.169244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:00.182226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:00.254034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 218/430] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:00.278169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e8cd-4f81-4899-8a95-b6c17e1a9a10 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:00.314195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689954d3-4260-45ea-af46-a4a112e02494 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:00.347246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:00.360424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:00.362400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.362694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.362974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.364262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:00.366698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:00.369438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 213/431] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:25:00 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:00.375054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:00.412869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e8cd-4f81-4899-8a95-b6c17e1a9a10 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:00.449874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:00.450719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.451004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.451311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.465569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:00.476548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:00.477545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:00.478413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 219/432] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:00.483972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:00.512534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:00.523894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d942e8-ba69-450a-94d0-1b72c37ce90a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:00.571372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:00.572161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.572473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.573212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.574853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:00.577906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:00.579691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 214/433] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:25:00 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:00.586322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:00.606612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7a883-fdc7-4c19-bf51-171dd8db2a4f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:00.649503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:00.650539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.651074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.651947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.653068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:00.653631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:00.655657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:00.655983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:00.667173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 220/434] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:00.677829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:00.698302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089d3c9-fe1c-4f29-a519-a217a50f2b32 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:00.741655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b2452-0b9f-43b2-a7f3-bd08b89ebd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:00.742415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b2452-0b9f-43b2-a7f3-bd08b89ebd73 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.742752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b2452-0b9f-43b2-a7f3-bd08b89ebd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.743056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b2452-0b9f-43b2-a7f3-bd08b89ebd73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.743908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b2452-0b9f-43b2-a7f3-bd08b89ebd73 None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=9cc91942b0de4b1bb86179386ef67e09, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:00.744239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b2452-0b9f-43b2-a7f3-bd08b89ebd73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:00.745906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b2452-0b9f-43b2-a7f3-bd08b89ebd73 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:00.753325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 215/435] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:25:00 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=9cc91942b0de4b1bb86179386ef67e09 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:00.795187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:00.808947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43164-7e17-43f9-92e6-19e911e58f05 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.809492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43164-7e17-43f9-92e6-19e911e58f05 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.809492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43164-7e17-43f9-92e6-19e911e58f05 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.810768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 216/436] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:00.817739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a913-7e90-4e03-83f7-6e1ede371814 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.818186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a913-7e90-4e03-83f7-6e1ede371814 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.818367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a913-7e90-4e03-83f7-6e1ede371814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.823772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041eff9-a87e-4832-9d27-f8ff6af7291a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:00.836692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4327a913-7e90-4e03-83f7-6e1ede371814 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:00.837893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4327a913-7e90-4e03-83f7-6e1ede371814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:00.875315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:00.876549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:00.878113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:00.878600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:00.926510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 217/437] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:00.950511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a913-7e90-4e03-83f7-6e1ede371814 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:00.965399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:00.966920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:01.038971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.080642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.104100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 221/438] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:01.109361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a913-7e90-4e03-83f7-6e1ede371814 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.109946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.150638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.151473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.151758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.152033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.168976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:01.177957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:01.178941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:01.179737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 218/439] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:01.186979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.254114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c3af5-4286-44ef-b582-dc62c63b0488 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.294347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d373f3-0aef-4984-b15d-83d06237fb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.295362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d373f3-0aef-4984-b15d-83d06237fb40 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.295718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d373f3-0aef-4984-b15d-83d06237fb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.296112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d373f3-0aef-4984-b15d-83d06237fb40 None admin] PATH_INFO: `/v3/projects/8efd660d626b4141b5c9be1824e7e416` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.297611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d373f3-0aef-4984-b15d-83d06237fb40 None admin] RBAC: Authorizing `identity:get_project(project_id=8efd660d626b4141b5c9be1824e7e416)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:01.300144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d373f3-0aef-4984-b15d-83d06237fb40 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:01.302779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 222/440] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/projects/8efd660d626b4141b5c9be1824e7e416 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:01.304239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ff61-c51d-4b67-88b6-de35ce77e60f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.304594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ff61-c51d-4b67-88b6-de35ce77e60f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.304960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ff61-c51d-4b67-88b6-de35ce77e60f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.306232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 223/441] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:01.309751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ff61-c51d-4b67-88b6-de35ce77e60f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.327505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbf22f-3cc4-4af6-b5df-bd3f66a42292 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.342791 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.363837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105a12c-2641-4263-b7e6-a8e64d638c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.383748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105a12c-2641-4263-b7e6-a8e64d638c7f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.383748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105a12c-2641-4263-b7e6-a8e64d638c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.383748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105a12c-2641-4263-b7e6-a8e64d638c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.384039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.384117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105a12c-2641-4263-b7e6-a8e64d638c7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:01.384117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105a12c-2641-4263-b7e6-a8e64d638c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:01.384117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105a12c-2641-4263-b7e6-a8e64d638c7f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:01.384117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 219/442] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:01.384117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b1cdc-0d69-43d8-b3a8-b7f5556a4ffa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.384117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b1cdc-0d69-43d8-b3a8-b7f5556a4ffa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.384450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b1cdc-0d69-43d8-b3a8-b7f5556a4ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.395674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-378b1cdc-0d69-43d8-b3a8-b7f5556a4ffa None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:01.396885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-378b1cdc-0d69-43d8-b3a8-b7f5556a4ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:01.432201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ff61-c51d-4b67-88b6-de35ce77e60f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.465343 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 220/443] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:01.466194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b1cdc-0d69-43d8-b3a8-b7f5556a4ffa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.466369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.467179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.467461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.467732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.469017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:01.471195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:01.473963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 224/444] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:01.478915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.568372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b1cdc-0d69-43d8-b3a8-b7f5556a4ffa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.575813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.581156 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a292e6c-07b9-4e3e-a8c2-d71e05aa2663 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.612205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.612374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.612374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.612374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.612374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.616718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.617435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.617715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.617988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.618815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=8efd660d626b4141b5c9be1824e7e416) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:01.619126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:01.620590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:01.626702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 225/445] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=8efd660d626b4141b5c9be1824e7e416&role.id=4f582d9851794ff894a232e045525ec4 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:01.627542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.679030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:01.680574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:01.750803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25060599-0697-4ae7-9e0b-5928d06247d1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.790451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.791257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.791553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.791832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.806464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:01.808467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 221/446] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:01.814731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:01.815340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:01.816232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:01.817080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 226/447] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:01.823200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.824862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:01.877891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.936079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d7004-2eb7-4e4f-9b21-492e217b196c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.961355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f23ad0-582a-4cd5-949f-ee7f328e86ba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:01.974661 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:01.975531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:01.975811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:01.976085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] PATH_INFO: `/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:01.977423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] RBAC: Authorizing `identity:get_project(project_id=bc6581cd5cb54c7aacb51e90736e34aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:01.979217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:01.980583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 222/448] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/projects/bc6581cd5cb54c7aacb51e90736e34aa => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:01.986731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:02.008525 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:02.009236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:02.010370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:02.010656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:02.011424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:02.011742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:02.013233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:02.022994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 227/449] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:02.028837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:02.078735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:02.121149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e79017-04e9-488e-9654-5e2e8d0b70da None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:02.161480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699621f6-28b2-45f7-b876-e8e0d28ad981 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:02.164667 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:02.165956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:02.166833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:02.168286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:02.168286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] PATH_INFO: `/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:02.169888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] RBAC: Authorizing `identity:get_group(group_id=1dfc3f7ccfb942a4a95d499b5e4e3a72)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:02.171372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:02.173864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 223/450] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:25:01 2022] GET /identity/v3/groups/1dfc3f7ccfb942a4a95d499b5e4e3a72 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:02.179271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:02.199849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:02.200505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:02.200897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:02.201083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:02.271274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:02.272753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:02.283805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012d6b1-7958-456c-a664-efde891003b5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:02.318165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:02.318826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:02.319110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:02.319388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:02.320215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] RBAC: Adding query filter params (group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72, role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:02.320525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:02.321993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:02.327963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 224/451] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:25:02 2022] GET /identity/v3/role_assignments?group.id=1dfc3f7ccfb942a4a95d499b5e4e3a72&scope.project.id=bc6581cd5cb54c7aacb51e90736e34aa&role.id=4f582d9851794ff894a232e045525ec4 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:02.374924 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 228/452] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:25:02.381301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:02.390643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:02.489882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d177b-1c96-426f-a38a-09b3550c61e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:02.524511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:02.525406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:02.525685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:02.525957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] PATH_INFO: `/v3/projects/233b60ec452b417c902a9645e05969db` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:02.527151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] RBAC: Authorizing `identity:get_project(project_id=233b60ec452b417c902a9645e05969db)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:02.528607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:02.530330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 225/453] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:25:02 2022] GET /identity/v3/projects/233b60ec452b417c902a9645e05969db => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:02.536716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:02.540495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:02.641831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a780e9-cb41-43b8-9c91-777fddf0d978 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:02.677187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5153802-86ad-4601-8e45-f56593c24423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:02.678047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5153802-86ad-4601-8e45-f56593c24423 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:02.678329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5153802-86ad-4601-8e45-f56593c24423 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:02.678601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5153802-86ad-4601-8e45-f56593c24423 None admin] PATH_INFO: `/v3/groups/5950e7ed89754f02890a194ed5c2dca7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:02.679962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5153802-86ad-4601-8e45-f56593c24423 None admin] RBAC: Authorizing `identity:get_group(group_id=5950e7ed89754f02890a194ed5c2dca7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:02.681464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5153802-86ad-4601-8e45-f56593c24423 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:02.683082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 229/454] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 18 10:25:02 2022] GET /identity/v3/groups/5950e7ed89754f02890a194ed5c2dca7 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:02.689773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:02.763815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:02.787949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0438e-db83-4243-8af2-5a75e5655b27 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:02.823562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf40ac-1717-4fb9-99eb-3d2205a9850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:02.824219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf40ac-1717-4fb9-99eb-3d2205a9850b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:02.824507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf40ac-1717-4fb9-99eb-3d2205a9850b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:02.824810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf40ac-1717-4fb9-99eb-3d2205a9850b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:02.825643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf40ac-1717-4fb9-99eb-3d2205a9850b None admin] RBAC: Adding query filter params (group.id=5950e7ed89754f02890a194ed5c2dca7, role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=233b60ec452b417c902a9645e05969db) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:02.825961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf40ac-1717-4fb9-99eb-3d2205a9850b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:02.827397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf40ac-1717-4fb9-99eb-3d2205a9850b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:02.833357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 226/455] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 18 10:25:02 2022] GET /identity/v3/role_assignments?group.id=5950e7ed89754f02890a194ed5c2dca7&scope.project.id=233b60ec452b417c902a9645e05969db&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:05.344504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e68ac-c8ed-4eac-a4cc-9ed083248223 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:05.345247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e68ac-c8ed-4eac-a4cc-9ed083248223 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:05.345247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e68ac-c8ed-4eac-a4cc-9ed083248223 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:05.346861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 230/456] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:05.352373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76adc0-1918-40ee-ba2a-51cadbd295cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:05.352887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76adc0-1918-40ee-ba2a-51cadbd295cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:05.352982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76adc0-1918-40ee-ba2a-51cadbd295cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:05.378933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ba76adc0-1918-40ee-ba2a-51cadbd295cf None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:05.379845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba76adc0-1918-40ee-ba2a-51cadbd295cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:05.479394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 227/457] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:05.520207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e68ac-c8ed-4eac-a4cc-9ed083248223 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:05.624032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22bda4-a5f1-48a4-873e-def4ee5a5cfa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:05.624643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22bda4-a5f1-48a4-873e-def4ee5a5cfa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:05.625174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22bda4-a5f1-48a4-873e-def4ee5a5cfa None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:05.628621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 228/458] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:05.633389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a3806-13be-4112-8692-0433ceb31d04 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:05.633960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a3806-13be-4112-8692-0433ceb31d04 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:05.634437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a3806-13be-4112-8692-0433ceb31d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:05.659948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-455a3806-13be-4112-8692-0433ceb31d04 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:05.661665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-455a3806-13be-4112-8692-0433ceb31d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:05.666114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:05.727644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e68ac-c8ed-4eac-a4cc-9ed083248223 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:05.748962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:05.817329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 229/459] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:05.817329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2abe1-651c-45b0-b101-80819128a538 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:05.817329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2abe1-651c-45b0-b101-80819128a538 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:05.817329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2abe1-651c-45b0-b101-80819128a538 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:05.817329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 230/460] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:05 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:05.817329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dea457-fa6c-4044-9dab-4eb3c0286f60 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:05.817329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dea457-fa6c-4044-9dab-4eb3c0286f60 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:05.818108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dea457-fa6c-4044-9dab-4eb3c0286f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:05.818183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:05.818183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:05.818183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:05.818183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:05.843831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c8dea457-fa6c-4044-9dab-4eb3c0286f60 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:05.845741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8dea457-fa6c-4044-9dab-4eb3c0286f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:05.885866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:05.887591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:05.959723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:05.960664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:05.961548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:05.961548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:05.961548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:05.961548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:05.970893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 231/461] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:05.973832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dea457-fa6c-4044-9dab-4eb3c0286f60 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:05.974888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 231/462] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:05.982123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:06.164920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dea457-fa6c-4044-9dab-4eb3c0286f60 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.199505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:06.210850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:06.212452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.213342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.213931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.233103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:06.235801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:06.253184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f7cb8-95bb-432b-ab07-5ce334dfd541 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:06.262238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:06.263130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:06.263965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:06.263965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:06.263965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:06.263965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:06.269352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 232/463] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:06.272609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:06.291782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:06.310821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d4411-319d-414c-9412-13c60beddb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:06.312194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d4411-319d-414c-9412-13c60beddb76 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.312942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d4411-319d-414c-9412-13c60beddb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.313385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d4411-319d-414c-9412-13c60beddb76 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.314857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d4411-319d-414c-9412-13c60beddb76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:06.315747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d4411-319d-414c-9412-13c60beddb76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:06.317753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d4411-319d-414c-9412-13c60beddb76 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:06.351207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 232/464] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:06.354295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309722-1f54-447c-a19d-945708abcd0f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.355343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309722-1f54-447c-a19d-945708abcd0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.355965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309722-1f54-447c-a19d-945708abcd0f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.364437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 233/465] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:06.365625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309722-1f54-447c-a19d-945708abcd0f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:06.402015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a030390-3c89-4c99-8555-9fe164f83063 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.441448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:06.443240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.443713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.444191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.456952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:06.458540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:06.477180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:06.478418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:06.480208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:06.480208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:06.480208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:06.480208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:06.480208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 233/466] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:06.480208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:06.490716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309722-1f54-447c-a19d-945708abcd0f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.513574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:06.531665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999984cc-5ad9-40a6-90ab-3d4637c7e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:06.533056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999984cc-5ad9-40a6-90ab-3d4637c7e5cd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.533465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999984cc-5ad9-40a6-90ab-3d4637c7e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.533747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999984cc-5ad9-40a6-90ab-3d4637c7e5cd None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.534561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999984cc-5ad9-40a6-90ab-3d4637c7e5cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:06.534833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999984cc-5ad9-40a6-90ab-3d4637c7e5cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:06.536228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999984cc-5ad9-40a6-90ab-3d4637c7e5cd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:06.547986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 234/467] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:06.552028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1757ce-52cc-456a-b5f7-1e6c525437c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.552028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1757ce-52cc-456a-b5f7-1e6c525437c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.552028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1757ce-52cc-456a-b5f7-1e6c525437c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.568373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf1757ce-52cc-456a-b5f7-1e6c525437c3 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:06.569919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf1757ce-52cc-456a-b5f7-1e6c525437c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:06.590987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8734037-0aba-454b-bcc1-250d452dd01e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.615253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:06.630068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:06.630747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.631034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.631311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.632030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:06.633510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:06.646120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 235/468] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:06.647176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1757ce-52cc-456a-b5f7-1e6c525437c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:06.768511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 234/469] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:25:06 2022] POST /identity/v3/users => generated 246 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:06.769416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:06.776202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1757ce-52cc-456a-b5f7-1e6c525437c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.824512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:06.825304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.825589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.825867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.826727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:06.826978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:06.828480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:06.848721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 236/470] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:06.848721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:06.891051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:06.930145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:06.944264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829c9f8-1ee8-4afd-b086-ddf802d12aff None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.985414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b385e1-a936-4716-b1eb-4289e353716f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:06.989743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da54b6-1424-4ba2-a2fb-5c1f7d651cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:06.990448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da54b6-1424-4ba2-a2fb-5c1f7d651cde None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:06.990729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da54b6-1424-4ba2-a2fb-5c1f7d651cde None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:06.991002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da54b6-1424-4ba2-a2fb-5c1f7d651cde None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:06.991716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da54b6-1424-4ba2-a2fb-5c1f7d651cde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:06.993208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da54b6-1424-4ba2-a2fb-5c1f7d651cde None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:07.025586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:07.026309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:07.026589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:07.026861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:07.040385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:07.041900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:07.061843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:07.063445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:07.065049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:07.065049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:07.065049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:07.065049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:07.065049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 237/471] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:07.065556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:07.127659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 235/472] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:25:06 2022] POST /identity/v3/users => generated 248 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:07.138850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8c77-10cd-4ca3-837e-00cef3f5e0a4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:07.139150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8c77-10cd-4ca3-837e-00cef3f5e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:07.139445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8c77-10cd-4ca3-837e-00cef3f5e0a4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:07.145716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 236/473] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:07.145716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb8c77-10cd-4ca3-837e-00cef3f5e0a4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:07.172076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:07.195340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcdcf7-2208-4811-96c4-c289ccd35db8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:07.245981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:07.247023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:07.247266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:07.247678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:07.248612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:07.250250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:07.291613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb8c77-10cd-4ca3-837e-00cef3f5e0a4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:07.331217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c20635-6744-49ab-ba0c-cd579944dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:07.331894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c20635-6744-49ab-ba0c-cd579944dac2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:07.332248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c20635-6744-49ab-ba0c-cd579944dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:07.332531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c20635-6744-49ab-ba0c-cd579944dac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:07.333309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c20635-6744-49ab-ba0c-cd579944dac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:07.333626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c20635-6744-49ab-ba0c-cd579944dac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:07.335003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c20635-6744-49ab-ba0c-cd579944dac2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:07.347927 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 237/474] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:07.349250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7d2e-219f-4dd9-abe1-574afde70bba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:07.349542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7d2e-219f-4dd9-abe1-574afde70bba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:07.349820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7d2e-219f-4dd9-abe1-574afde70bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:07.355722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:07.367553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-863c7d2e-219f-4dd9-abe1-574afde70bba None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:07.368808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-863c7d2e-219f-4dd9-abe1-574afde70bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:07.377166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 238/475] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:25:07 2022] POST /identity/v3/users => generated 248 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:07.381428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:07.452632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 238/476] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:07.524237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c7d2e-219f-4dd9-abe1-574afde70bba None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:07.643385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf0059-7709-4e05-917d-f55ad2bb0e56 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:07.662399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c7d2e-219f-4dd9-abe1-574afde70bba None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:07.684500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:07.685170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:07.685505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:07.685783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:07.686503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:07.690642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:07.709307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:07.710519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:07.716302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:07.716302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:07.716302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:07.716302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:07.716302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:07.904311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.sql.core [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:25:07.904311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 10:25:07.904311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [parameters: {'id': 'f188be440ca74ebf819b97ed9ef5e699', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 10:25:07.904311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 18 10:25:07.922592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:25:07.924346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application yield resource Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise self.value Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application session.commit() Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:25:07.926460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:25:07.928178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:25:07.931026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:25:07.942478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [parameters: {'id': 'f188be440ca74ebf819b97ed9ef5e699', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:07.943613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 10:25:07.944577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:07.945883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 239/477] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 18 10:25:07 2022] POST /identity/v3/roles => generated 144 bytes in 401 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:07.945883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:07.982534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:07.999689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 239/478] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:25:07 2022] POST /identity/v3/users => generated 249 bytes in 621 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:08.084217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb476a81-8f09-4b90-86e2-5b4df4b97a2a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:08.126881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09f62f-c953-4c97-955f-bdee6aba2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:08.127816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09f62f-c953-4c97-955f-bdee6aba2168 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:08.128174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09f62f-c953-4c97-955f-bdee6aba2168 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:08.128520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09f62f-c953-4c97-955f-bdee6aba2168 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:08.142360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09f62f-c953-4c97-955f-bdee6aba2168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5f09f62f-c953-4c97-955f-bdee6aba2168 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:08.151091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:08.152308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:25:08.153327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:08.159247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 240/479] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:08.160501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:08.327110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cacc55-808b-497a-a6c6-1b1f1b1d042f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:08.368326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43848c-c0a0-47ba-90d9-5385644c15d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:08.369027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43848c-c0a0-47ba-90d9-5385644c15d4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:08.369310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43848c-c0a0-47ba-90d9-5385644c15d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:08.369584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43848c-c0a0-47ba-90d9-5385644c15d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:08.370346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43848c-c0a0-47ba-90d9-5385644c15d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:08.370651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43848c-c0a0-47ba-90d9-5385644c15d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:08.372036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43848c-c0a0-47ba-90d9-5385644c15d4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:08.388942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 240/480] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:09.102577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126e23-57a2-428e-a44a-433e0ff80c93 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:09.103062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126e23-57a2-428e-a44a-433e0ff80c93 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:09.103062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126e23-57a2-428e-a44a-433e0ff80c93 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:09.104690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 241/481] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:09.110746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f6186-22f6-4dcf-beb5-a7cc40a7db91 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:09.111041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f6186-22f6-4dcf-beb5-a7cc40a7db91 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:09.111399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f6186-22f6-4dcf-beb5-a7cc40a7db91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:09.136285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-401f6186-22f6-4dcf-beb5-a7cc40a7db91 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:09.137591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-401f6186-22f6-4dcf-beb5-a7cc40a7db91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:09.213100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:09.226745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 241/482] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:09.252693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126e23-57a2-428e-a44a-433e0ff80c93 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:09.330531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:09.376047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126e23-57a2-428e-a44a-433e0ff80c93 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:09.413343 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d395ee5-81bb-465b-a7b3-09bc3aa2339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:09.414145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d395ee5-81bb-465b-a7b3-09bc3aa2339e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:09.414491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d395ee5-81bb-465b-a7b3-09bc3aa2339e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:09.414795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d395ee5-81bb-465b-a7b3-09bc3aa2339e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:09.426845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d395ee5-81bb-465b-a7b3-09bc3aa2339e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:09.433282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e54fdd-6490-424e-aa98-46692174850d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:09.433759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e54fdd-6490-424e-aa98-46692174850d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:09.433833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e54fdd-6490-424e-aa98-46692174850d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:09.435185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 242/483] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7d395ee5-81bb-465b-a7b3-09bc3aa2339e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:09.439226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:09.440148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:09.441011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:09.444927 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 242/484] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:09.446735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e54fdd-6490-424e-aa98-46692174850d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:09.451990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35913bbf-8df0-40a9-8e62-c5e08d47eef0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:09.452281 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35913bbf-8df0-40a9-8e62-c5e08d47eef0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:09.452557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35913bbf-8df0-40a9-8e62-c5e08d47eef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:09.489468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-35913bbf-8df0-40a9-8e62-c5e08d47eef0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:09.490683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35913bbf-8df0-40a9-8e62-c5e08d47eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:09.612067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 243/485] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:09.635139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35913bbf-8df0-40a9-8e62-c5e08d47eef0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:09.655413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e54fdd-6490-424e-aa98-46692174850d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:09.714686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:09.721965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:09.722655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:09.722942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:09.723221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:09.723987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:09.724298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:09.734070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:09.744117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 243/486] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:09.762078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:09.778488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35913bbf-8df0-40a9-8e62-c5e08d47eef0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:09.856511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:09.857904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:09.858397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:09.858889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:09.867828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:09.884123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:09.895106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:09.896108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:09.897032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:09.910056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 244/487] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:09.916318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:09.958556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d3d5d-d461-4f70-ba61-b21ae234ad54 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.000431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c6632-bcb3-4fd6-9686-d1e05ee92bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.001346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c6632-bcb3-4fd6-9686-d1e05ee92bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.001631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c6632-bcb3-4fd6-9686-d1e05ee92bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.001897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c6632-bcb3-4fd6-9686-d1e05ee92bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.013707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c6632-bcb3-4fd6-9686-d1e05ee92bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.015270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c6632-bcb3-4fd6-9686-d1e05ee92bc2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d18c6632-bcb3-4fd6-9686-d1e05ee92bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:10.035144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:10.036078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:10.037368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:10.037368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:10.037368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:10.037368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:10.037368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 244/488] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:10.037994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4d52-dff3-4183-aee5-27af03edacdc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.038287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4d52-dff3-4183-aee5-27af03edacdc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.038570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4d52-dff3-4183-aee5-27af03edacdc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.040188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 245/489] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:10.044516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e4d52-dff3-4183-aee5-27af03edacdc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.092906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fe5d1-6e5f-4519-a7be-c649a112f73b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.105172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:10.133352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9b457-5578-4f06-9108-ab4787ef4b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.134033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9b457-5578-4f06-9108-ab4787ef4b33 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.134675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9b457-5578-4f06-9108-ab4787ef4b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.134970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9b457-5578-4f06-9108-ab4787ef4b33 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.135742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9b457-5578-4f06-9108-ab4787ef4b33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:10.136054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9b457-5578-4f06-9108-ab4787ef4b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.138124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9b457-5578-4f06-9108-ab4787ef4b33 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.154269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 245/490] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:10.156033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb86d47-b140-4c36-9cd0-d96fc9281303 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.156534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb86d47-b140-4c36-9cd0-d96fc9281303 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.157708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb86d47-b140-4c36-9cd0-d96fc9281303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.168213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e4d52-dff3-4183-aee5-27af03edacdc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.171024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:10.178363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5eb86d47-b140-4c36-9cd0-d96fc9281303 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:10.182184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5eb86d47-b140-4c36-9cd0-d96fc9281303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:10.214408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.215026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.215307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.215585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.216318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:10.216721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.218160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.233452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 246/491] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:10.234708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.263086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 246/492] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:10.264289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb86d47-b140-4c36-9cd0-d96fc9281303 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.362986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9622928-e41b-416b-b84a-369ccd88818f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.384625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:10.388156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb86d47-b140-4c36-9cd0-d96fc9281303 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.401082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.405376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.405376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.405376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.418370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.420177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.425611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.426405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.426687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.426961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.435819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.437299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:10.442235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:10.443412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:10.444251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:10.444251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:10.444251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:10.444251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:10.446623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 247/493] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:10.447781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:10.454824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:10.455848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:10.457316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:10.462828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 247/494] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:10.462828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.482654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:10.575361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849badb-8bd9-4cab-8224-fc2cc86c2d42 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.577590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e364e8a-448e-4a7e-947a-3ac4300e51e6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.614021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.614266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.614685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.614964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.615250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.615998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:10.616295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.616671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.617199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.617700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.617798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.629908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:10.631806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 248/495] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:10.632447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:10.639613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:10.640533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:10.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:10.644593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 248/496] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:10.645632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.662102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:10.729058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:10.753475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ad9e6-f9dd-4388-989d-2befa841cfcf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.753655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191516c1-3f7b-4f20-823a-74de104e5e0c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.790160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.791097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:10.791623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.791747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:10.792037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.792313 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.792640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:10.793163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:10.793272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:10.793678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.795781 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.815941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:10.818370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:10.837374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:10.838316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:25:10.839213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:10.842133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 249/497] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:10.842472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 249/498] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:10.843540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.849004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:10.944650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:10.967878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ebc69-b35b-4b2b-9a89-f1775ed904e9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:10.976615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81948610-d6ca-401e-b1a5-18d33606a7b4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.009568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73dec1-abb3-47fe-ad71-b029f4c13e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.010353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73dec1-abb3-47fe-ad71-b029f4c13e7a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.010678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73dec1-abb3-47fe-ad71-b029f4c13e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.010913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73dec1-abb3-47fe-ad71-b029f4c13e7a None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.011754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73dec1-abb3-47fe-ad71-b029f4c13e7a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:11.012265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73dec1-abb3-47fe-ad71-b029f4c13e7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.013364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.013755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73dec1-abb3-47fe-ad71-b029f4c13e7a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.014092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.014349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.014770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.015812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:11.016669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.019086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.021545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.028617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 250/499] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:11.029790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.034998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 250/500] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 18 10:25:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:11.036418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa45bba-da6c-4844-9c5f-fe897da43a93 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.036721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa45bba-da6c-4844-9c5f-fe897da43a93 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.037052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa45bba-da6c-4844-9c5f-fe897da43a93 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.038565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 251/501] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:11.042021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165b8c4-22ec-4abd-8080-ebcd0220733b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.042313 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165b8c4-22ec-4abd-8080-ebcd0220733b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.046429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165b8c4-22ec-4abd-8080-ebcd0220733b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.046429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 252/502] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:11.046429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165b8c4-22ec-4abd-8080-ebcd0220733b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.136442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0628c-8a92-4f22-8c9d-e7a7be7ab0d6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.150343 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165b8c4-22ec-4abd-8080-ebcd0220733b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.175033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba33142-d7fa-4c01-a660-4fa41c6bbaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.175683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba33142-d7fa-4c01-a660-4fa41c6bbaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.175967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba33142-d7fa-4c01-a660-4fa41c6bbaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.176242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba33142-d7fa-4c01-a660-4fa41c6bbaa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.185182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.185964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.186244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.186802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.189790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.197651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.199029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:11.217163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:11.218376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:11.219239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:11.219239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:11.219239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.219239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:11.219239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 253/503] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:11.219239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.249354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.253059 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba33142-d7fa-4c01-a660-4fa41c6bbaa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.254532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba33142-d7fa-4c01-a660-4fa41c6bbaa0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.323947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588e748-f45b-40ab-95a5-1ab5cf8143ef None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.356274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 251/504] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:11.357750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8fbc3-35b4-4878-9d15-aa50ba442651 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.358042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8fbc3-35b4-4878-9d15-aa50ba442651 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.358322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8fbc3-35b4-4878-9d15-aa50ba442651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.363545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.364197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.364485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.364762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.376179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-afe8fbc3-35b4-4878-9d15-aa50ba442651 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:11.377410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-afe8fbc3-35b4-4878-9d15-aa50ba442651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:11.423962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.436400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.439598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.445464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 252/505] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:11.446721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a547-a9c3-478a-8976-bffa87993a9e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.447012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a547-a9c3-478a-8976-bffa87993a9e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.447285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a547-a9c3-478a-8976-bffa87993a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.465812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-88a2a547-a9c3-478a-8976-bffa87993a9e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:11.466950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-88a2a547-a9c3-478a-8976-bffa87993a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:11.490949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.537435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 253/506] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:11.538287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a547-a9c3-478a-8976-bffa87993a9e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.546317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 254/507] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:11.547141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.635104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.641808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a547-a9c3-478a-8976-bffa87993a9e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.647624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ceb0f-485a-4cf7-93a4-a7a6b782b698 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.676628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.677350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.677630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.677906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.678634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:11.678942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.680399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.692476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 254/508] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:11.693339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.694404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.695392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.695847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.696124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.707368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.708885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.713766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:11.729462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:11.730336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:11.731248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:11.731248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:11.731248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.731248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:11.731248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 255/509] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:11.731554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.802772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833fe18-46d1-4926-aeaf-5cc14ebf3b41 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.834119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f63f7c-e8a2-4e88-a442-1901c4db0fb8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:11.836836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.837632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.837910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.838183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.849758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.851263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:11.868469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:11.869911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:11.870779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:11.870779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:11.870779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.870779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:11.871002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:11.871089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 255/510] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:11.871089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.871471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:11.871834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:11.872120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:11.883200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.885638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:11.887088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:11.910426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:11.911935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:11.914074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:11.914074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:11.914074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:11.914074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:11.914074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 256/511] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:11.914074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:11.950549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:11.984289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5bd22-2327-4b1b-b3bd-ab881d97e268 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.020633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.021397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c4e47-6189-4a80-bb6a-b14605bb1f9e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.021505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.021727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.022008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.032975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:12.040760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:12.041714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:12.042541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 256/512] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:12.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.061941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.062939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.063229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.063512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.071093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.072554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:12.088402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:12.089346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:12.090211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 257/513] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:12.090827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.146509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:12.150908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6d040-0edd-467d-abbd-92749d930bb8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.179439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:12.185158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.185815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.186099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.186373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.187092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:12.187401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.189321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.194617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e76b76-f951-4060-b701-948f6b777506 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.200739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 257/514] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:12.201608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.232927 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.233498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.233783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.234061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.234872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:12.235677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.237132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.249361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 258/515] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:12.250970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.306129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f16d790-2c1f-412d-a2e6-f1c83f00db04 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.341627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.342279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.342556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.342830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.343544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:12.343847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.345998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.357048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 258/516] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:12.357847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.358209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21369-246e-4013-8653-48c929f4a37b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.372583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:12.395121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.395795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.396082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.396363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.397151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:12.397465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.398877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.407740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 259/517] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:12.408539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.413487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:12.464071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bd47f-3337-4f84-91c7-fad3919890d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.498302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.498896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.499186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.499463 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.501122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:12.501574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.503076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.508376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3d698-a10b-40e6-9a32-b7b85952a005 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.540306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 259/518] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:12.541343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.543612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.544527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.544950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.545228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.552958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.554438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:12.569336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:12.570484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 260/519] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:12.571435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.605171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:12.647166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc818f-c482-4ad2-8854-3f5cd4be6737 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.657868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:12.674637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fe919-f410-4d02-aed9-865902ec48e9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.682873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.683549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.683829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.684105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.684899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.686470 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.709147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.709926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.710207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.710479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.717189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.718677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:12.733768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:12.734639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:12.735760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 261/520] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:12.736273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.829176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:12.835567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acb68d-aa7f-4444-9198-63a8ae24ca32 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:12.869608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:12.870436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:12.870736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:12.871012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:12.873316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 260/521] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:25:12 2022] POST /identity/v3/users => generated 247 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:12.874060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:12.883197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:12.884831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:12.906607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:12.908328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:12.909586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:12.909586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:12.909586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:12.909586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:12.909586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 262/522] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:12.909892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.012747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8cfd-5d09-4a1a-b6b4-f88437b38847 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.054826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c17098-680c-4432-8ce6-46fec6c6f125 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.055048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.055563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.055852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.056128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.098166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.098880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.099271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.099550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.100101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:13.101213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:13.101534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.103013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.124835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 263/523] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:13.124835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.134167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:13.150066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.153351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.247041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616303fd-8c72-4d7c-ba03-6fa8b497b73d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.260251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 261/524] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:13.261320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.288907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.289670 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.290406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.290406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.291622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:13.292121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.294232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.313264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 264/525] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:13.313264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.341564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:13.385831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788c07-7d81-41bc-97de-6639d1abfaf2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.386449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:13.423902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c315a8-9651-461f-a109-56e5c739dfe9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.425745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.426522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.426910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.427208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.427985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:13.428296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.429736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.443110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 262/526] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:13.443930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.469015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.470133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.470433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.470704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.471551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=ec899ae414744dde9ec9b76aac1ce9d8) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:13.471869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.473347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.488814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 265/527] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/role_assignments?user.id=ec899ae414744dde9ec9b76aac1ce9d8&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:13.488814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.575810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e01a0-0d66-4502-a069-3bfbd916d8e0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.602641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:13.615864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.623456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.623456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.623456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.628634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532039de-6141-4342-8020-2de5224bbe4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.632121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.633764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:13.654491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:13.655997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:13.662337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:13.662337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:13.662337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:13.662337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:13.662337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 263/528] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:13.663388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.668072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:13.675897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.677050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.677555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.678001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.679144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=62e462a1b125428f9893459d5d965ced) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:13.679462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.682237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.688606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 266/529] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/role_assignments?user.id=62e462a1b125428f9893459d5d965ced&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:13.689853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.857088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a73f09-f014-4c0e-8421-2b21c94dc523 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.898369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.899039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.899326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.899602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:13.900341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:13.900647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:13.902051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:13.914881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8751e2-6a62-4395-9817-c343dc5184c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:13.930153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 267/530] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:13.935608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:13.983074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:13.983878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:13.984159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:13.984445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:14.002456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:14.003961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:14.034187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:14.035170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:14.036432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 264/531] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:14.041105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:14.052511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:14.120206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0b2bf-9f69-403c-8011-e3976c0dc08c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:14.137552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:14.193076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:14.193886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:14.194168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:14.194441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:14.210101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:14.211587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:14.242580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:14.243699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:14.244881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:14.246209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 268/532] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:14.252663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:14.265277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239b343-23eb-48c6-bdf3-263f94e0695a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:14.318743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:14.319889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:14.320317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:14.320736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:14.321745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:14.322216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:14.324027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:14.344345 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 265/533] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Jul 18 10:25:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:14.358897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:14.377776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b778a-39c1-4ffb-ba45-7e461e04d7cb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:14.393519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:14.415422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:14.416084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:14.416367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:14.416639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:14.417583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:14.417895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:14.419513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:14.432763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 269/534] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:14.441936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:14.498934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d0da-cd5e-4718-96aa-4f8656127235 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:14.513436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:14.538315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:14.539340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:14.539780 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:14.540290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:14.615940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:14.618248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:14.651778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddef3e2-716c-4ab8-aec9-f6a965f2f015 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:14.706238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:14.706900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:14.707237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:14.707777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:14.708357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=80535e7da62e4790931fd74000653411) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:14.708672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:14.710180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:14.716733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 270/535] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:14 2022] GET /identity/v3/role_assignments?user.id=80535e7da62e4790931fd74000653411&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:14.730447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:14.731099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 266/536] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:25:14.741970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:14.825382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:14.925377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7c32b-ef1c-4927-8356-a2861a10e9ab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:14.977139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfaa67-8200-418e-ad28-1a12a0363c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:14.977965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfaa67-8200-418e-ad28-1a12a0363c32 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:14.978254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfaa67-8200-418e-ad28-1a12a0363c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:14.978526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfaa67-8200-418e-ad28-1a12a0363c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:14.991573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfaa67-8200-418e-ad28-1a12a0363c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:14.993095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfaa67-8200-418e-ad28-1a12a0363c32 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3ddfaa67-8200-418e-ad28-1a12a0363c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:15.012642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:15.013557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:15.014861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:15.014861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:15.014861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:15.014861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:15.014861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 271/537] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:15.020655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:15.156874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736c252-64a7-490d-99ad-6b99b2d6dda1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:15.195812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:15.196761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:15.197255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:15.197377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1ef31-db35-46b7-87ce-76a8973c4daf None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:15.197584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1ef31-db35-46b7-87ce-76a8973c4daf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:15.197886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1ef31-db35-46b7-87ce-76a8973c4daf None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:15.198084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:15.198974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:15.199426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:15.199615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 272/538] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:15.201368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:15.205583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72901991-8998-45dd-aec4-418828523942 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:15.205890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72901991-8998-45dd-aec4-418828523942 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:15.206210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72901991-8998-45dd-aec4-418828523942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:15.215853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 267/539] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:15.222632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:15.231821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-72901991-8998-45dd-aec4-418828523942 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:15.240052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-72901991-8998-45dd-aec4-418828523942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:15.341285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:15.366923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 273/540] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:15.399076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72901991-8998-45dd-aec4-418828523942 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:15.426905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:15.450042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d5ef2-c241-4a9e-a057-2600410898fd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:15.538664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:15.539881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:15.540358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:15.540846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:15.557252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:15.559064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:15.585689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:15.586612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:15.587456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:15.592850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72901991-8998-45dd-aec4-418828523942 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:15.602854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 268/541] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:15.608706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:15.655137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:15.657847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:15.659935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:15.670777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:15.690164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:15.698351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:15.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:15.700131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:15.708236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 274/542] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:15.714845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:15.800370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f372c-b658-4c7c-8c2c-8f78f66af225 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:15.820695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:15.849614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:15.853916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:15.854447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:15.854925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:15.855928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:15.856454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:15.860706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:15.896519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 269/543] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:15.902615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:15.904709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:15.928474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6bc70-6ebf-4cc5-abb3-ed394031a0b5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:15.981903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976ad3e-67b1-43bf-9a3d-e38d7c9edab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:15.982924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ad3e-67b1-43bf-9a3d-e38d7c9edab5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:15.983403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ad3e-67b1-43bf-9a3d-e38d7c9edab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:15.983869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ad3e-67b1-43bf-9a3d-e38d7c9edab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:15.985024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976ad3e-67b1-43bf-9a3d-e38d7c9edab5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:15.985842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976ad3e-67b1-43bf-9a3d-e38d7c9edab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:15.987445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976ad3e-67b1-43bf-9a3d-e38d7c9edab5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:16.002895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 275/544] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:16.004578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179ecf-d52a-4779-b89f-6d082320a70c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.005089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179ecf-d52a-4779-b89f-6d082320a70c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.005596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179ecf-d52a-4779-b89f-6d082320a70c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.007295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 276/545] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:16.013080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25605be-e1fb-49e2-9476-e78e3ed4c5c9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.013658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25605be-e1fb-49e2-9476-e78e3ed4c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.014138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25605be-e1fb-49e2-9476-e78e3ed4c5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.033482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b25605be-e1fb-49e2-9476-e78e3ed4c5c9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:16.035032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b25605be-e1fb-49e2-9476-e78e3ed4c5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:16.048671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac5b88-35f7-43fb-960b-cb048bfc1182 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.086241 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.086894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.087180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.087455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.088289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=a48b0fdacb2c43649916304e1237f39c) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:16.088602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:16.090102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:16.096854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 270/546] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:15 2022] GET /identity/v3/role_assignments?user.id=a48b0fdacb2c43649916304e1237f39c&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:16.097029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.122064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 277/547] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:16.125578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e12a8-cd3b-40a6-bea6-034666c37862 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.126260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e12a8-cd3b-40a6-bea6-034666c37862 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.127139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e12a8-cd3b-40a6-bea6-034666c37862 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.129509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 278/548] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:16.137430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc43787-ab37-41ac-92f3-b239b4c5c064 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.137430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc43787-ab37-41ac-92f3-b239b4c5c064 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.137430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc43787-ab37-41ac-92f3-b239b4c5c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.159437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:16.161794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4bc43787-ab37-41ac-92f3-b239b4c5c064 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:16.163094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4bc43787-ab37-41ac-92f3-b239b4c5c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:16.237365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd77c-7e04-4994-8359-5ab0730cc4d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.264932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 279/549] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:16.266486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc43787-ab37-41ac-92f3-b239b4c5c064 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.269675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:16.289411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.293787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.293787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.293787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.304536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:16.307125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:16.328316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:16.329774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:16.330824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:16.330824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:16.330824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:16.330824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:16.330824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 271/550] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:16.330824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.394574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc43787-ab37-41ac-92f3-b239b4c5c064 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.436102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:16.438676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.439688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.440021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.440324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.446752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65585e9e-2e73-4423-b8a5-2c5294fe1c37 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.452042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:16.460719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:16.461660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:16.462509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 280/551] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:16.463356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.484876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.485774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.486197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.486542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.498222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:16.506971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:16.509127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:16.511107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:16.520821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 272/552] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:16.520821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.536236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:16.581026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316f684-8d81-482e-9ea8-0c7ad0818279 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.620203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.621019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.621435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.621716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.622568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:16.622944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:16.624396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:16.625477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d875-2409-4051-957c-82cffc8e3a61 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.638739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 281/553] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:16.639712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.662839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.663510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.663793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.664067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.665501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:16.665501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:16.666774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:16.676469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 273/554] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:16.677417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.694058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:16.805068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce3c98-a4f7-4fed-aca8-29767cf6ed1f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.830198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:16.841906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d6ac5-6d37-4df4-a4ce-7755ec54f7cc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:16.843067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.844581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.844992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.846246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.847038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:16.847423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:16.852254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:16.863691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 282/555] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:16.864860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.882290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:16.883122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:16.883615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:16.883886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:16.895023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:16.897334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:16.918260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:16.920163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:16.921304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:16.925022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 274/556] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:16.927300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:16.987735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcdbb9-5ec1-4274-bb29-0e8849e7ec76 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.010590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.025402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.026207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.026493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.027059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.039426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.040935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.048602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec839d-8ef6-4cd8-8a29-a99c906235c0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:17.059188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:17.060536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:17.061450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:17.061450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:17.061450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:17.061450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:17.061450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 283/557] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:17.061729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.090720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.092042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.092599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.093112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.099865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.107288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.109102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:17.128398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:17.129605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:17.130580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:17.130580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:17.130580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:17.130580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:17.132726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 275/558] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:17.133756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.176336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347da856-c5fc-4198-bb57-bff3fc927a34 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.212546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.213377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.213664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.213996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.214933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:17.215246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.216959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.237784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 284/559] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:17.242437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.258564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c914b-ca74-4d30-a146-bfbe5695222d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.298460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.316411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.317466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.317750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.318317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.319060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:17.319368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.320799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.336037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 276/560] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:17.337038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.373384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.382920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642466e-9811-47db-98db-5c54b4d70d28 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.420681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.421452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.421945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.422238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.422979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:17.423287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.424671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.438228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 285/561] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:17.439102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.480697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2702fe8-6f97-4a5f-87f5-c40018c36b57 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.530799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.532304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.532844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.533346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.552907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0f16-fd1d-499e-9692-3a2c0c45ba3d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.590253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee157b-c90c-4c26-898d-e3bec3a691cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.591002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee157b-c90c-4c26-898d-e3bec3a691cd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.591311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee157b-c90c-4c26-898d-e3bec3a691cd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.591611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee157b-c90c-4c26-898d-e3bec3a691cd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.600519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee157b-c90c-4c26-898d-e3bec3a691cd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.602005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee157b-c90c-4c26-898d-e3bec3a691cd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.606623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-08ee157b-c90c-4c26-898d-e3bec3a691cd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:17.617015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:17.625052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 286/562] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d588-b8ba-4e3c-b6b6-ecf547770695 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.626986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d588-b8ba-4e3c-b6b6-ecf547770695 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.627683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d588-b8ba-4e3c-b6b6-ecf547770695 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.627683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 287/563] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:17.627683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2d588-b8ba-4e3c-b6b6-ecf547770695 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.635174 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.636806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.642809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.738249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2d588-b8ba-4e3c-b6b6-ecf547770695 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.754142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 277/564] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:17.754142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.782860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaee7eb-fe0f-4601-9277-3f7d9adb97ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.783713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaee7eb-fe0f-4601-9277-3f7d9adb97ee None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.784027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaee7eb-fe0f-4601-9277-3f7d9adb97ee None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.784304 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaee7eb-fe0f-4601-9277-3f7d9adb97ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.792949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaee7eb-fe0f-4601-9277-3f7d9adb97ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.794523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaee7eb-fe0f-4601-9277-3f7d9adb97ee None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-bfaee7eb-fe0f-4601-9277-3f7d9adb97ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:17.816895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:17.817911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:17.818866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 288/565] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:17.819988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a965d3-bfbd-4fcf-b9cc-d6ff97e5aa69 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.819988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a965d3-bfbd-4fcf-b9cc-d6ff97e5aa69 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.819988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a965d3-bfbd-4fcf-b9cc-d6ff97e5aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.837878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a1a965d3-bfbd-4fcf-b9cc-d6ff97e5aa69 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:17.839133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1a965d3-bfbd-4fcf-b9cc-d6ff97e5aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:17.855934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.876532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e4c0c-289d-4565-a9e6-9125ff9f808b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:17.924825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 289/566] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:17.924825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a965d3-bfbd-4fcf-b9cc-d6ff97e5aa69 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:17.930141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:17.936983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:17.937825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:17.938382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:17.938684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:17.939532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:17.939839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:17.941482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:17.962228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 278/567] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:17.963139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.052935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a965d3-bfbd-4fcf-b9cc-d6ff97e5aa69 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.077787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e08f6-ab6b-4074-8841-299e665a26d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.088715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.089557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.089838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.090110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.100890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.102373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.116312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.117260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.117770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.118220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.119224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:18.119704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:18.120721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:18.121740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:18.126348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:18.126348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:18.126348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.126348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:18.126348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 290/568] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:18.126348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.128235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.145917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.154566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 279/569] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:25:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:18.154566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.192551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.237225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04292d11-e39b-40e4-8e2a-5c0cb165a023 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.270073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b921c-0e0f-42b3-8d27-70025f6a6916 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.273181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.273898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.274168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.274443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.285574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:18.293324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:18.295126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:18.296388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:18.297624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 291/570] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:18.298042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.311232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.312020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.312398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.312896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.324243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.325783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:18.343305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:18.344726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:18.345734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:18.345734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:18.345734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.345734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:18.345734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 280/571] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:18.346169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.400161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.403071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bdd12-77c0-4aa8-b439-8f6d102858a2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.421898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.441019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b70fdb-43ae-434f-8c01-b04d8412fa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.441692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b70fdb-43ae-434f-8c01-b04d8412fa5d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.442055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b70fdb-43ae-434f-8c01-b04d8412fa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.442318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b70fdb-43ae-434f-8c01-b04d8412fa5d None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.442980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b70fdb-43ae-434f-8c01-b04d8412fa5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:18.443290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b70fdb-43ae-434f-8c01-b04d8412fa5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.444662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b70fdb-43ae-434f-8c01-b04d8412fa5d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.448406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c3ef4-aff1-4763-a47c-c4a4a01ca36e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.461368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 292/572] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:18.461368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c78bb1-6d65-4e78-b1ce-e26f95922c70 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.461368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c78bb1-6d65-4e78-b1ce-e26f95922c70 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.461368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c78bb1-6d65-4e78-b1ce-e26f95922c70 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.461368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 293/573] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:18.461368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c78bb1-6d65-4e78-b1ce-e26f95922c70 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.489883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.490616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.490909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.491188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.503067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.504565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:18.523693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:18.525240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:18.527009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:18.527009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:18.527009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.527009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:18.527009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 281/574] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:18.527009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.577281 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c78bb1-6d65-4e78-b1ce-e26f95922c70 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.620611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.621314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.621603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.621878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.622627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:18.622934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.624307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.633576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 294/575] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:18.634376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.645108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d7f81-2e72-455a-95f3-26d81ddd867f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.647567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.676890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.680968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1722b-b8cf-4ff1-b45a-33f40e03f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.681627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1722b-b8cf-4ff1-b45a-33f40e03f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.681905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1722b-b8cf-4ff1-b45a-33f40e03f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.682175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1722b-b8cf-4ff1-b45a-33f40e03f9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.682904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1722b-b8cf-4ff1-b45a-33f40e03f9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:18.683203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1722b-b8cf-4ff1-b45a-33f40e03f9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.684589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1722b-b8cf-4ff1-b45a-33f40e03f9e5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.697067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 282/576] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:18.698312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588221ea-89dd-4c97-873a-ef798321582d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.698596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588221ea-89dd-4c97-873a-ef798321582d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.698876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588221ea-89dd-4c97-873a-ef798321582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.714947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-588221ea-89dd-4c97-873a-ef798321582d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:18.716132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-588221ea-89dd-4c97-873a-ef798321582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:18.740538 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c55f0c-7ff1-4051-ab9b-e59677f9b3e8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.775466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.776353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.776634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.776942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.782680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 283/577] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:18.783529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588221ea-89dd-4c97-873a-ef798321582d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.784137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.785605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:18.800553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:18.801551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:18.802574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 295/578] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:18.803156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.894130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.900833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588221ea-89dd-4c97-873a-ef798321582d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.902819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:18.906949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9b74f-b45d-4df0-8e7d-b0649ba805f8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:18.939650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.940332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.940929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.941425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.941533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:18.942302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:18.942484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:18.942665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:18.942790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.942894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:18.944582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.953711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:18.955203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:18.960915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 284/579] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:18.960915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:18.973526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:18.974469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:18.975380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:18.975380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:18.975380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:18.975380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:18.975380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 296/580] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:18.975664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.068200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577acde-1cd4-4ac6-af63-02663c91962e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.078875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a460f-c91d-4acd-8915-e0abbbc02456 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.103415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.104213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.104496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.104791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.112408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.113507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.113898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.114299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.114588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.114861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.125795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:19.128389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:19.129439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 285/581] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:19.130426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:19.133629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:19.134642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:19.135489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:19.137045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 297/582] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:19.137820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.151451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.152351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.239716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cf00-5795-44f6-9529-18f4971b1aa5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.243674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59360234-9c11-4199-93d5-8e0384d6c07b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.274377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.275036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.275317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.275592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.276400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:19.276711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.278218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.279399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.280192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.280476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.280761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.288368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.289898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.290274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 286/583] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:19.291053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:19.308802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:19.309733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:19.310765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 298/584] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:19.311430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.370752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.384768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.395742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3607-c858-450d-abc3-ea47f7d1a0d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.420205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd9a4-ff5e-4a87-940e-118241401097 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.429284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.429944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.430227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.430504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.431277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:19.431692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.433168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.446028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 287/585] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:19.446855 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.456571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.457408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.457694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.457967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.458762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:19.459074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.460543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.474585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 299/586] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:19.475419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.555208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaec5f-38e2-4e32-a114-e0502efcfb9f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.580116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b181d-baa1-4cdc-874b-66bcdbde70c7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.589233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.589895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.590180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.590451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.591203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:19.591500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.592935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.594849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.601615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 288/587] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:19.602411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.609450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.615457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48564629-be42-4490-80e2-c53495776e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.616498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48564629-be42-4490-80e2-c53495776e5d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.616498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48564629-be42-4490-80e2-c53495776e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.621092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48564629-be42-4490-80e2-c53495776e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.621092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48564629-be42-4490-80e2-c53495776e5d None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=642a42337be14f10b410903df57f7527) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:19.621092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48564629-be42-4490-80e2-c53495776e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.621092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48564629-be42-4490-80e2-c53495776e5d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.626102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 300/588] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/role_assignments?user.id=642a42337be14f10b410903df57f7527&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:19.626933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48564629-be42-4490-80e2-c53495776e5d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.717705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080421e5-cae7-427a-9b0a-7c488a9ff8ec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.766934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.767337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48564629-be42-4490-80e2-c53495776e5d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.767610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.767894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.768246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.769011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:19.769325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.770916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.785034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 289/589] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:19.786261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.816546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.817445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.818003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.818121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.826741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.829074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:19.844710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:19.846086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:19.847105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 301/590] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:19.848010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:19.864447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.913702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9708c-1610-45d7-9256-5e3d40ba5a4e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.915367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:19.958094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:19.959349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:19.959675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:19.959949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/62e462a1b125428f9893459d5d965ced/roles/30ebd4c6517a4a9ca23f5eaf73c732a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:19.968053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed93366-a83e-4a9c-971e-0237a303f5a7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:19.976730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=62e462a1b125428f9893459d5d965ced, role_id=30ebd4c6517a4a9ca23f5eaf73c732a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:19.978236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.004215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.005050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.005336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.005611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.017052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.018494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:20.036535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:20.043224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:20.044141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:20.044141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:20.044141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:20.044141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:20.044141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 302/591] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:20.044141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:20.084893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 290/592] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:19 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/62e462a1b125428f9893459d5d965ced/roles/30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:20.084893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:20.194788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f552a-b671-44a6-8ba2-759eee99affa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:20.205562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:20.222139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108a5da-09af-4e1b-9be8-7122fe5da0d6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:20.230787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:20.242345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.243274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.243533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.243805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/ec899ae414744dde9ec9b76aac1ce9d8/roles/30ebd4c6517a4a9ca23f5eaf73c732a0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.262999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=ec899ae414744dde9ec9b76aac1ce9d8, role_id=30ebd4c6517a4a9ca23f5eaf73c732a0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.264111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.264816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.265800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.266411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.267018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.268171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:20.270289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.270289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.289704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 291/593] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 18 10:25:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:20.290749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:20.358620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 303/594] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:20 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/ec899ae414744dde9ec9b76aac1ce9d8/roles/30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:20.365894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:20.413120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f072cb-93c0-44e9-83fb-6c4b9ecac46d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:20.460807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.460807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.460807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.460807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.461254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:20.461254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.461688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.476973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 292/595] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:20.487201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:20.561105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:20.607961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912951-cbc2-42bd-bc6b-2762c883a96b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:20.657423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.659301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.659821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.660310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.660950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa7df-06e7-40f4-872b-eb46c2057b39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:20.676461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.679284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:20.702634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:20.703899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:20.704838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:20.704838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:20.704838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:20.704838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:20.707657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 304/596] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:20.718155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:20.721596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.722506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.722927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.723198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.734299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:20.745656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.747150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:20.778215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:20.779272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:25:20.780238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:20.789380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 293/597] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:20.799012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:20.909769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:20.911671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f64e3-cf95-4343-9b31-3ba208aaf005 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:20.927445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197362b-f95e-4e5f-94b6-a23fdf8097d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:20.965523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.966489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.966927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.967366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.968477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:20.969441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.971075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.990916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 294/598] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Jul 18 10:25:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:20.995336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:20.995336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:20.995336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:20.995336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:20.995744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:20.995744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:20.996192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:20.997301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:21.009946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 305/599] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:21.019128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:21.115835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:21.169417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa82d3-14a6-49a0-a04e-b767986d0800 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:21.208705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18687c6f-4cf6-45b0-b693-ba12a95063c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:21.224240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:21.225103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.225395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.225671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.239068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:21.240549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:21.268369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:21.269199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.269480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.269755 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:21.277376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:21.278465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:21.279384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 306/600] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:21.281513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:21.282997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:21.284967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:21.285366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:21.304804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:21.308941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:21.310004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:21.310004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:21.310004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:21.310004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:21.310004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 295/601] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:21.312800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:21.457522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003a47-d0bc-40eb-a922-f04f274d2a57 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:21.504058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:21.505046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.505475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.506034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.507701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:21.508334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:21.510900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:21.517726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e540df-57d3-4766-b4a0-94b6491dbd78 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:21.538835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 296/602] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:21.541604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:21.590174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:21.605972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:21.608929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:21.609615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.609971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.610185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.611011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:21.611511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:21.612802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:21.645112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 307/603] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:21.646179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:21.657784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd711fe-f05d-4f7f-b549-b3f09ef14c54 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:21.702866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:21.704040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.704610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.705330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.715010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:21.716862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:21.749557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:21.751211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:21.752147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:21.755051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 297/604] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:21.761885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:21.784346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8a9a9-67f7-46ff-8e40-e698c218050d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:21.865248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996e9e8-c3ca-475f-9348-0e808330fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:21.866462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996e9e8-c3ca-475f-9348-0e808330fce2 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.867440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996e9e8-c3ca-475f-9348-0e808330fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.867739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996e9e8-c3ca-475f-9348-0e808330fce2 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/80535e7da62e4790931fd74000653411/roles/30ebd4c6517a4a9ca23f5eaf73c732a0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.891529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996e9e8-c3ca-475f-9348-0e808330fce2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=80535e7da62e4790931fd74000653411, role_id=30ebd4c6517a4a9ca23f5eaf73c732a0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:21.893064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996e9e8-c3ca-475f-9348-0e808330fce2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:21.914513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00d914-125d-4944-87a5-14dcced24e18 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:21.931172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:21.954050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61372b63-d970-46e8-9e04-4044c4e7bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:21.954722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372b63-d970-46e8-9e04-4044c4e7bc70 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.955004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372b63-d970-46e8-9e04-4044c4e7bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.955275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372b63-d970-46e8-9e04-4044c4e7bc70 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.956074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61372b63-d970-46e8-9e04-4044c4e7bc70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:21.956382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61372b63-d970-46e8-9e04-4044c4e7bc70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:21.957927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61372b63-d970-46e8-9e04-4044c4e7bc70 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:21.975676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 298/605] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:21.975676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2e92-bb3f-47cd-b9f8-f64100f18e0a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:21.975676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2e92-bb3f-47cd-b9f8-f64100f18e0a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:21.975676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2e92-bb3f-47cd-b9f8-f64100f18e0a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:21.978643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 299/606] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:21.979419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2e92-bb3f-47cd-b9f8-f64100f18e0a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:22.037658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 308/607] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:21 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/80535e7da62e4790931fd74000653411/roles/30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:22.039045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942e776-ed24-4a93-85ba-9a124019c291 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.039338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942e776-ed24-4a93-85ba-9a124019c291 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.039611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942e776-ed24-4a93-85ba-9a124019c291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.094899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7942e776-ed24-4a93-85ba-9a124019c291 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:22.096120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7942e776-ed24-4a93-85ba-9a124019c291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:22.112491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2e92-bb3f-47cd-b9f8-f64100f18e0a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:22.166796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:22.167710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.168104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.168467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/a48b0fdacb2c43649916304e1237f39c/roles/30ebd4c6517a4a9ca23f5eaf73c732a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.175155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:22.207275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=a48b0fdacb2c43649916304e1237f39c, role_id=30ebd4c6517a4a9ca23f5eaf73c732a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:22.208814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:22.224799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 309/608] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:22.248898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942e776-ed24-4a93-85ba-9a124019c291 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:22.336275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 300/609] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:21 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/a48b0fdacb2c43649916304e1237f39c/roles/30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:22.372736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942e776-ed24-4a93-85ba-9a124019c291 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:22.419886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895439c8-18c8-43e8-a102-4e11240bdd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:22.420746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895439c8-18c8-43e8-a102-4e11240bdd2b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.421268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895439c8-18c8-43e8-a102-4e11240bdd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.421543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895439c8-18c8-43e8-a102-4e11240bdd2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.434558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895439c8-18c8-43e8-a102-4e11240bdd2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-895439c8-18c8-43e8-a102-4e11240bdd2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:22.444007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:22.445061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:22.446360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 310/610] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:22.452101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:22.490675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:22.502943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df4b6-9f61-4b9a-a2f3-e510601c96f4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.503135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df4b6-9f61-4b9a-a2f3-e510601c96f4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.503407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df4b6-9f61-4b9a-a2f3-e510601c96f4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.505111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 311/611] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:22.512442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea478fa0-ea61-4da2-a7e8-fd223d6ac777 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.513081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea478fa0-ea61-4da2-a7e8-fd223d6ac777 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.513528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea478fa0-ea61-4da2-a7e8-fd223d6ac777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.556744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ea478fa0-ea61-4da2-a7e8-fd223d6ac777 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:22.558359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea478fa0-ea61-4da2-a7e8-fd223d6ac777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:22.585358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:22.589454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbeab06-b064-4df3-bf3d-eb4fdfba647d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:22.629917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:22.630639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.630922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.631195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.631952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:22.632261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:22.633729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:22.638295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 312/612] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:22.645111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 301/613] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:22.650790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea478fa0-ea61-4da2-a7e8-fd223d6ac777 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:22.665348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:22.769783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:22.789518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e48798-6e7d-429b-9580-ab0a2d1a3070 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:22.793997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea478fa0-ea61-4da2-a7e8-fd223d6ac777 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:22.832953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:22.834367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.835019 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.835513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.858942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:22.860225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0548f2-71b6-413c-931d-e46c2cfd6d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:22.865707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0548f2-71b6-413c-931d-e46c2cfd6d1d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.866002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0548f2-71b6-413c-931d-e46c2cfd6d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.866286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0548f2-71b6-413c-931d-e46c2cfd6d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:22.869183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:22.870110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:22.870976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:22.873722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 302/614] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:22.881178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:22.899594 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0548f2-71b6-413c-931d-e46c2cfd6d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:22.902902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0548f2-71b6-413c-931d-e46c2cfd6d1d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-bd0548f2-71b6-413c-931d-e46c2cfd6d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:22.930653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:22.931655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:22.932545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:22.932545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:22.932545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:22.932545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:22.932545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 313/615] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:22.933618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53bc9-74b1-4931-86e0-a45e1e202f51 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:22.933908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53bc9-74b1-4931-86e0-a45e1e202f51 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:22.934188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53bc9-74b1-4931-86e0-a45e1e202f51 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:22.935625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 314/616] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:22.939296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c53bc9-74b1-4931-86e0-a45e1e202f51 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:22.958638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.023531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8544d5-4c4a-4950-953d-fa9cb834db5c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.062897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ae30f-eed5-4607-b121-a90eea90a3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.063678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ae30f-eed5-4607-b121-a90eea90a3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.063974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ae30f-eed5-4607-b121-a90eea90a3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.064261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ae30f-eed5-4607-b121-a90eea90a3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.065316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ae30f-eed5-4607-b121-a90eea90a3c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:23.065502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ae30f-eed5-4607-b121-a90eea90a3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.066945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ae30f-eed5-4607-b121-a90eea90a3c1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:23.077378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c53bc9-74b1-4931-86e0-a45e1e202f51 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.078446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 303/617] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:23.084284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f124-10a3-4af3-a91a-4a8ae553b7f6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.084284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f124-10a3-4af3-a91a-4a8ae553b7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.084284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f124-10a3-4af3-a91a-4a8ae553b7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.102891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6561f124-10a3-4af3-a91a-4a8ae553b7f6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:23.104169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6561f124-10a3-4af3-a91a-4a8ae553b7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:23.114235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.128022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.129051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.129781 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.130164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.131199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:23.131514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.133703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:23.153016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 315/618] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:23.154806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.190848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 304/619] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:23.192107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561f124-10a3-4af3-a91a-4a8ae553b7f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.266417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.310987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcee0f8-35a2-4ec1-9fac-2a2df7d87ed7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.314099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561f124-10a3-4af3-a91a-4a8ae553b7f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.355585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.356977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.357465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.357741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.358257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.358492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.358808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.359086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.367906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.369835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:23.372146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.372334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.374996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:23.388014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:23.395968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:23.396979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:23.397545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:23.398346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:23.399255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:23.399255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:23.399255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:23.399255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:23.400252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 316/620] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:23.400434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.400952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 305/621] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:23.402034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.521727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd219a-3dcd-4140-9185-64e48ee975eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.540157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f17a0d-10ee-4c7d-80c8-42dd4fd6a3d2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.558831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.559910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.560405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.560964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.562201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:23.562739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.564499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:23.571486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.588901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 306/622] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:23.590158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.594515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.594515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.594515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.594515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.607689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:23.618038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:23.619008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:23.619840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 317/623] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:23.626968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.678496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.738938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5fcf9-9498-4a9f-b5b0-ab506cc6eb71 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.745799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a98ab0-44bc-440a-a21f-fd431dec53e1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.782807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.783466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.783846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.784047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.784752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:23.785090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.786514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:23.794890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.796451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.797386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.797875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.803546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 307/624] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:23.803546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.817346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.819327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:23.845573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:23.846708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:23.847705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:23.847705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:23.847705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:23.847705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:23.847705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 318/625] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:23.852814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:23.886255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.938636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8271c-79d4-4e7a-aa09-971db2a6861d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.961475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa664a-c38b-4940-b247-7af6306f252b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:23.962475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:23.990665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:23.991752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:23.992607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:23.993017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:23.994176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:23.994830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:23.997167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.003067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.003951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.004391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.004715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.012886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 308/626] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:24.013814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.014014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.016038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:24.033077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:24.035229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:25:24.037421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:24.045067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 319/627] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:24.045067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.143495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a636d48-c44e-477a-9520-4a83f91a5134 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.148202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:24.156179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144f19-978d-4abf-839b-7f37060dbff5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.194391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.195092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.195381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.195655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.196389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:24.196697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.197186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.198121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.198426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.198903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.199385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.213130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.214932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.216148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 309/628] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:24.217111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:24.238829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:24.239807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:24.245589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:24.245589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:24.245589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:24.245589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:24.245589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 320/629] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:24.245589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.273447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:24.338775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb93ea-0334-47d5-a856-64d8c3a8bbb6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.377493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.378163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.378443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.378715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.379560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:24.379866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.381093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b6ade-3843-4bea-8fbd-3efa1dbf2595 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.381419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.405204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 310/630] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:24.407003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.425937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.426941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.427348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.427755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.437846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.439499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:24.455336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:24.456283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:24.457216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:24.461275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 321/631] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:24.462059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.473462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:24.535005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:24.542344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ed6dd-631b-4e4b-b649-276da648a8f3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.576254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efeebe-d02e-4e02-ac9e-f6c2ce7cda6b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.596175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.596864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.597151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.597472 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.598256 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:24.598581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.601046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.619053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 311/632] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:24.619053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.621357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfed82b-c2b6-4f20-8f0a-1b2535a9f517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.622030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfed82b-c2b6-4f20-8f0a-1b2535a9f517 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.622313 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfed82b-c2b6-4f20-8f0a-1b2535a9f517 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.622591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfed82b-c2b6-4f20-8f0a-1b2535a9f517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.704065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfed82b-c2b6-4f20-8f0a-1b2535a9f517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.705667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfed82b-c2b6-4f20-8f0a-1b2535a9f517 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.715799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:24.740081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbee0b3-eaec-4609-80c3-8bec82590794 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.777834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3931858-fc9b-46e2-bc1f-7206ab3defce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.778500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931858-fc9b-46e2-bc1f-7206ab3defce None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.778784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931858-fc9b-46e2-bc1f-7206ab3defce None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.779063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931858-fc9b-46e2-bc1f-7206ab3defce None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.779860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931858-fc9b-46e2-bc1f-7206ab3defce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:24.780166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931858-fc9b-46e2-bc1f-7206ab3defce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.782385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931858-fc9b-46e2-bc1f-7206ab3defce None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.795117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 312/633] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:24.796650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f03d38-49f7-4356-a09a-2a11b311f9ca None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.796976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f03d38-49f7-4356-a09a-2a11b311f9ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.797310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f03d38-49f7-4356-a09a-2a11b311f9ca None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.798945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 313/634] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:24.803815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f03d38-49f7-4356-a09a-2a11b311f9ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.823835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 322/635] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:24.825264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fc864-5acb-4e98-8aaa-dcbfbbe0f2de None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.825559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fc864-5acb-4e98-8aaa-dcbfbbe0f2de None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.825843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fc864-5acb-4e98-8aaa-dcbfbbe0f2de None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.827153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 323/636] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:24.828370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:24.828629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fc864-5acb-4e98-8aaa-dcbfbbe0f2de None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:24.933449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f03d38-49f7-4356-a09a-2a11b311f9ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.950423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fc864-5acb-4e98-8aaa-dcbfbbe0f2de None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:24.959876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:24.977837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f1c0d-3ffc-4b66-8a4e-f503ef4e7e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.980218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f1c0d-3ffc-4b66-8a4e-f503ef4e7e0e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.980218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f1c0d-3ffc-4b66-8a4e-f503ef4e7e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.980218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f1c0d-3ffc-4b66-8a4e-f503ef4e7e0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:24.987041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f1c0d-3ffc-4b66-8a4e-f503ef4e7e0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:24.988513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f1c0d-3ffc-4b66-8a4e-f503ef4e7e0e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:24.991625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:24.992545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:24.992894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:24.993186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/642a42337be14f10b410903df57f7527/roles/30ebd4c6517a4a9ca23f5eaf73c732a0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-760f1c0d-3ffc-4b66-8a4e-f503ef4e7e0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:25.004616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:25.005850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:25.006802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 314/637] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:25:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:25.007423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56803c3-afd2-4e19-9fbc-8f9015131b0f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.007501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56803c3-afd2-4e19-9fbc-8f9015131b0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.007946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56803c3-afd2-4e19-9fbc-8f9015131b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.012595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=642a42337be14f10b410903df57f7527, role_id=30ebd4c6517a4a9ca23f5eaf73c732a0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:25.014086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:25.025916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a56803c3-afd2-4e19-9fbc-8f9015131b0f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:25.029092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a56803c3-afd2-4e19-9fbc-8f9015131b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:25.090069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 324/638] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:24 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/642a42337be14f10b410903df57f7527/roles/30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:25.090978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.117709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 315/639] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:25.120355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61745-8adc-4655-8835-30871efdfe6b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.120743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61745-8adc-4655-8835-30871efdfe6b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.121046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61745-8adc-4655-8835-30871efdfe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.138478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:25.149298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-26d61745-8adc-4655-8835-30871efdfe6b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:25.150584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26d61745-8adc-4655-8835-30871efdfe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:25.231337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0ce28-9811-4f79-820f-ea9ab5f44b9e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:25.250373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 316/640] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:25.250373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d61745-8adc-4655-8835-30871efdfe6b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.282432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:25.283273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.283602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.283893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.295486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:25.298210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:25.321377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:25.322772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:25.323993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:25.323993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:25.323993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:25.323993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:25.323993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 325/641] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:25.323993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.338444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:25.386562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d61745-8adc-4655-8835-30871efdfe6b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:25.426242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:25.426930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.427223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.427528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.428441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:25.429096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:25.431435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:25.435730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:25.454882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75ff4-3ffc-4ef6-8faa-23081b555a2c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:25.460532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 317/642] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:25.460532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.507446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:25.508706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.509224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.509689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.523331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:25.532635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:25.535254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:25.536161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 326/643] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:25.538229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.601092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47971-358b-44ba-a253-ce7cf3be84b9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:25.627455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:25.652422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:25.653305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.653598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.653913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.668394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:25.669813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f804-457e-40a7-b529-fcfe7ddffee6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:25.677165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:25.678233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:25.679128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 318/644] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:25.681036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.713481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:25.714497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.714961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.715416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.716356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:25.717063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:25.719302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:25.738556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 327/645] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:25.739482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.744102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:25.794519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf41ae-4f41-44cd-acf4-d6534ea5d27d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:25.837332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:25.837990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.838276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.838547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.858689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0e509-5b23-4fea-a72b-f8f47a847c07 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:25.890623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:25.898712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:25.899392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:25.899676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:25.899951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:25.900720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:25.901404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:25.902811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:25.911634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 328/646] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:25.912570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:25.923179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:25.924832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:25.988830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.025569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f5af-bd22-4a3b-988d-5073367083f0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.039162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 319/647] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:26.044286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.065243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.065845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.066133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.066410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.067163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:26.067466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.068856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.077412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 329/648] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:26.078523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.143228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.163194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da1321-58b2-4bbe-8eb2-bbba374c3a88 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.200434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.201274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.201561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.201840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.203243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993236-022b-4414-a13f-b3d0bf5cb445 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.208946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.210421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:26.224894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:26.225946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 320/649] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:26.226864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.243129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.243968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.244423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.244529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.249329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.258383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.260235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:26.277836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:26.278827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:26.279778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:26.279778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:26.279778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:26.279778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:26.280652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 330/650] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:26.281003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.344281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a099-8e20-4182-8836-377e52ead86f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.383856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.384666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.384984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.385263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.396711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.398218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.402183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3713d-6c61-4715-820f-c6df92b03855 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.404631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:26.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:26.417171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:26.418066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:26.418066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:26.418066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:26.418066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:26.418066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 321/651] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:26.418340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.444463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.445311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.445599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.445895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.457138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.458600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:26.479100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:26.480015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:26.481036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:26.481036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:26.481036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:26.481036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:26.481036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 331/652] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:26.481943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.494510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.533583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd196e7-b738-4ec4-98cd-e5f9adab74a9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.569145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.569811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.570113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.570365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.571150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:26.571460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.572930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.585637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 322/653] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:26.590731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.601922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbbe36-9a9a-45f2-a591-180736751595 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.638549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.639213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.639502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.640489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.641350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:26.641666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.643063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.660049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 332/654] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:26.660049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.679906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.691502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdefd8-b2c3-4b64-a27f-723440fa2e39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.720797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.729713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.730375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.730662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.730937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.731658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:26.731963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.733479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.745874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 323/655] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:26.746509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.776550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4185-3e9b-46df-8e44-76520af8d870 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.819728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.820649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.821005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.821301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.822038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:26.822349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.823764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.838287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 333/656] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:26.839263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.876362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf4183-3844-4d8a-a816-f8456ea9b146 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:26.916565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:26.917673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:26.917673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:26.917847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:26.918813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=62e462a1b125428f9893459d5d965ced) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:26.919228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:26.920688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:26.926019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:26.936805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 324/657] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/role_assignments?user.id=62e462a1b125428f9893459d5d965ced&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:26.936805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:26.970458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631f87-67d5-4d21-9544-ad54780c80a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.001038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:27.021720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.022542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.022826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.024875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.032850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.034349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:27.051208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:27.052227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:25:27.053192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:27.057534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 334/658] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:27.058481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.072124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69360a-d7f9-4cd1-97d7-3cce8cce0bf6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.129615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.130762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.131270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.131703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.142543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.144332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:27.160037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:27.161013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:27.161932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:27.163986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 325/659] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:27.165429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.182904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6b1ae-72b9-4b3c-85df-9b55fd579787 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.227489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.228309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.228595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.228931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.232682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:27.239853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.241624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:27.259722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:27.260733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:27.261921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 335/660] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:27.262552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.292709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad64ad-9227-4dff-b531-15d7dae7d2aa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.305337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:27.331793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.332465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.332752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.333050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.333894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:27.334208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.335735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.356937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 326/661] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:27.356937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.378278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ad40a-59fd-4ec4-a20a-02e10b0de445 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.415336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df6f54-accd-41bc-a190-88910ccd15d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.416049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6f54-accd-41bc-a190-88910ccd15d5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.416390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6f54-accd-41bc-a190-88910ccd15d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.416744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6f54-accd-41bc-a190-88910ccd15d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.417650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df6f54-accd-41bc-a190-88910ccd15d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:27.418029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df6f54-accd-41bc-a190-88910ccd15d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.419549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df6f54-accd-41bc-a190-88910ccd15d5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.437450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 336/662] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:27.439751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4648d5-18a8-40e3-9722-174eb29c6527 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.440051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4648d5-18a8-40e3-9722-174eb29c6527 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.440332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4648d5-18a8-40e3-9722-174eb29c6527 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.442718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 337/663] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:27.442718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4648d5-18a8-40e3-9722-174eb29c6527 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.478344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbc45f-52b3-4ee0-8703-dd3f40c4f790 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.506954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:27.523341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.524228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.524506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.524810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.528124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:27.528124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.528124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.550746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 327/664] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:27.551031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.558001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4648d5-18a8-40e3-9722-174eb29c6527 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.585392 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:27.599271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.600032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.600373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.600711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.661980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061c856-430b-42c9-9f29-2cea61bd97f6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.675482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.677008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.698220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b1dac-3e12-46ff-9f62-ea8350bcc970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:27.698976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b1dac-3e12-46ff-9f62-ea8350bcc970 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.699344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b1dac-3e12-46ff-9f62-ea8350bcc970 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.699727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b1dac-3e12-46ff-9f62-ea8350bcc970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.700676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b1dac-3e12-46ff-9f62-ea8350bcc970 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=ec899ae414744dde9ec9b76aac1ce9d8) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:27.701141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b1dac-3e12-46ff-9f62-ea8350bcc970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:27.703006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b1dac-3e12-46ff-9f62-ea8350bcc970 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:27.709317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 328/665] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/role_assignments?user.id=ec899ae414744dde9ec9b76aac1ce9d8&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:27.710620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd898-d4c7-4264-879c-21b1d28b02f5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:27.710968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd898-d4c7-4264-879c-21b1d28b02f5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:27.711369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd898-d4c7-4264-879c-21b1d28b02f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:27.736006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-30bbd898-d4c7-4264-879c-21b1d28b02f5 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:27.737821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-30bbd898-d4c7-4264-879c-21b1d28b02f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:27.752889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:27.819966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 338/666] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:27.821282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.848802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 329/667] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:27.848802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbd898-d4c7-4264-879c-21b1d28b02f5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:27.867776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:27.973356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71434-8fb0-46ff-a3c2-80073db69d74 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:27.978104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbd898-d4c7-4264-879c-21b1d28b02f5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.020792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.021470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.021758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.022039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.025871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.027291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.027291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.027291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.043149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.044654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.063426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:28.070995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:28.071937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:28.072879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:28.072879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:28.072879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:28.072879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:28.072879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 330/668] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:28.073591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.125144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.126784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.181208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:28.226580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574a573-97e1-4ada-80d2-dcdab7548990 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.247439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 339/669] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:28.248362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.278069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.279187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.279499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.280175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.295528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:28.305594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:28.307381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 331/670] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:28.309360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.323716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:28.379609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3cfb7-f586-46a1-ac0d-4b9d68a62c36 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.427461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.428433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.429128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.429432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.430443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:28.430769 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.432439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.438536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e3729-db32-4d45-a473-0306106fbe31 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.453467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 340/671] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:28.453467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.477790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:28.485360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.486163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.486446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.486717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.502227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.504294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:28.527912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:28.528922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:28.529895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:28.529895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:28.529895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:28.529895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:28.532849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 332/672] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:28.533976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.583286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50982f-4b34-4b20-8c59-4e8d961c9929 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.604447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:28.627632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.628310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.628597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.628902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.629922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:28.630063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.631518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.641345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 341/673] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:28.647258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.657788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27a66e-3f87-40eb-8aba-5da2c60ee37f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.695287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.696480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.696988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.697460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.706061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.710901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:28.723733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:28.724665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:28.725595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:28.732537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 333/674] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:28.734008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.766418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9963e0-d0ae-4101-aee2-708854eb7b46 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.789376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:28.813713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.814760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.815294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.815771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.816884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:28.817400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.819300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.835389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 342/675] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:28.836569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:28.879708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926208-5403-4d0b-9235-a5fdace29e01 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.887112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:28.932796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:28.933852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:28.934497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:28.934907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:28.951334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:28.953027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe51bd-90cd-45e5-b937-4ef2f3c505d3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:28.953228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:28.978096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:28.979259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:28.980132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:28.980132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:28.980132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:28.980132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:28.985779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 334/676] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:28.985779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.000824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.002165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.002666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.003133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.004179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:29.004766 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.006437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.027071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 343/677] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:29.028433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.111867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:29.118072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aecbfc-e8fe-4fee-8f4b-c71e6417930d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.145147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:29.149003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac79a-6436-4f46-a6c6-8b9b429ebdc9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.161300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.162644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.163104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.163680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.173679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.176189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.184861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ea2e6-5203-480d-b427-1a93f2477384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.185540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ea2e6-5203-480d-b427-1a93f2477384 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.185826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ea2e6-5203-480d-b427-1a93f2477384 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.186202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ea2e6-5203-480d-b427-1a93f2477384 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.187427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ea2e6-5203-480d-b427-1a93f2477384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:29.187747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ea2e6-5203-480d-b427-1a93f2477384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.189254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ea2e6-5203-480d-b427-1a93f2477384 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:29.191773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:29.193105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:29.194431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:29.197821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 335/678] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:29.198584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.203552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 344/679] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:29.204912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe05635-0d01-4a5e-b9b4-ff6f2fed9106 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.205207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe05635-0d01-4a5e-b9b4-ff6f2fed9106 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.205492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe05635-0d01-4a5e-b9b4-ff6f2fed9106 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.206872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 345/680] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:29.209577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe05635-0d01-4a5e-b9b4-ff6f2fed9106 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.320273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe05635-0d01-4a5e-b9b4-ff6f2fed9106 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.337032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8187-2992-4eed-b008-57c753a9a1fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.355611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.356278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.356569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.356875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.357676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:29.357991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.359504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.371327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 346/681] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:29.372147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.379292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8885fa-30e2-43d6-a318-aa4c34b81992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.380211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8885fa-30e2-43d6-a318-aa4c34b81992 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.380497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8885fa-30e2-43d6-a318-aa4c34b81992 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.380793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8885fa-30e2-43d6-a318-aa4c34b81992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.381679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8885fa-30e2-43d6-a318-aa4c34b81992 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=80535e7da62e4790931fd74000653411) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:29.381994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8885fa-30e2-43d6-a318-aa4c34b81992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.383432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8885fa-30e2-43d6-a318-aa4c34b81992 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.396644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:29.404480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 336/682] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/role_assignments?user.id=80535e7da62e4790931fd74000653411&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:29.404480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8c0b2-c965-4dcf-a7ac-8ea0c2818fc9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.404480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8c0b2-c965-4dcf-a7ac-8ea0c2818fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.404480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8c0b2-c965-4dcf-a7ac-8ea0c2818fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.423585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-44e8c0b2-c965-4dcf-a7ac-8ea0c2818fc9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:29.425374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-44e8c0b2-c965-4dcf-a7ac-8ea0c2818fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:29.442915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:29.500916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f57dd-306d-4622-bc48-dcbe9c47b429 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.515003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 337/683] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:29.516440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8c0b2-c965-4dcf-a7ac-8ea0c2818fc9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.541831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.542652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.542937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.543210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.552759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.554399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:29.571550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:29.572513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 347/684] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:29.574534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.663840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8c0b2-c965-4dcf-a7ac-8ea0c2818fc9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.681326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:29.707464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.708242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.708597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.708966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.709877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=a48b0fdacb2c43649916304e1237f39c) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:29.710193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.711662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.716003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584bc95-ee92-4a19-89e2-9c8b83407e24 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.718627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 338/685] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/role_assignments?user.id=a48b0fdacb2c43649916304e1237f39c&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:29.719548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.744211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:29.783587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.784440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.784724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.785046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.785732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.792771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:29.825127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:29.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:29.827133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 348/686] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:29.834493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:29.928092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cb99b-535e-405d-8f75-27ace3e953ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:29.986477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41552d3-3290-4265-a015-e6022574a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:29.987164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41552d3-3290-4265-a015-e6022574a678 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:29.987447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41552d3-3290-4265-a015-e6022574a678 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:29.987720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41552d3-3290-4265-a015-e6022574a678 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:29.988702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41552d3-3290-4265-a015-e6022574a678 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:29.989027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41552d3-3290-4265-a015-e6022574a678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:29.990522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41552d3-3290-4265-a015-e6022574a678 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:30.012513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 339/687] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:30.013958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603b439-2f0c-4856-818d-95ae91309fe3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.014761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603b439-2f0c-4856-818d-95ae91309fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.015264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603b439-2f0c-4856-818d-95ae91309fe3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.017031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 340/688] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:30.018900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603b439-2f0c-4856-818d-95ae91309fe3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.034250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbfcad-5c33-4927-b29a-8da74e45f75f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.089183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef25668-b25f-4379-a2a9-9a932a6c67b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.090037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef25668-b25f-4379-a2a9-9a932a6c67b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.091273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef25668-b25f-4379-a2a9-9a932a6c67b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.091656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef25668-b25f-4379-a2a9-9a932a6c67b6 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.092427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef25668-b25f-4379-a2a9-9a932a6c67b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:30.094031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef25668-b25f-4379-a2a9-9a932a6c67b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:30.095470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef25668-b25f-4379-a2a9-9a932a6c67b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:30.101726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 349/689] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 18 10:25:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:30.103740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654756c-8f9e-40dc-a010-451ccd8986a9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.104032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654756c-8f9e-40dc-a010-451ccd8986a9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.104308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654756c-8f9e-40dc-a010-451ccd8986a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.119474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:30.129113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1654756c-8f9e-40dc-a010-451ccd8986a9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:30.130461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1654756c-8f9e-40dc-a010-451ccd8986a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:30.137370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:30.177781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603b439-2f0c-4856-818d-95ae91309fe3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.222866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.224195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.224452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 350/690] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:30.224964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.225381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654756c-8f9e-40dc-a010-451ccd8986a9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.225735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.330414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:30.332888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:30.365979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654756c-8f9e-40dc-a010-451ccd8986a9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.405898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.406940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.407427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.407921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.408876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:30.409406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:30.410992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:30.414418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:30.418653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 351/691] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:30.419906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.425748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:30.476512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 341/692] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:30.481929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.569657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bc8f2-4c35-483f-b97f-216a53cb6acd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.607356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.609089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.609735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.610223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.622442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:30.640090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:30.641148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:30.641991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:30.643252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fa5cc-acb3-4574-8c4c-751801e4dfb1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.649660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 352/693] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:30.651033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.689484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.690294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.690595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.690869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.703684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:30.705209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:30.721445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:30.723609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:30.730379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:30.731420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:30.731420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:30.731420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:30.731420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:30.731420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 342/694] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:30.731420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.760577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:30.767964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505813-5069-4222-881d-dd3a721b5bfa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.821046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.821777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.822248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.822606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.823934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:30.824531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:30.826274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:30.839300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 353/695] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:30.845129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.854873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565ed5-d914-4c65-9882-3d33c96afa68 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.953590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.954319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.954670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.954961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:30.955712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:30.956028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:30.957490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:30.958633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a619a4a-3cef-48cb-9f65-1481361c17db None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:30.984041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 343/696] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:30.985132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:30.989600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:30.996316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:30.997148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:30.998371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:30.999568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.011801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.013297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:31.033355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:31.034305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:31.035603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:31.035603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:31.035603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:31.035603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:31.035603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 354/697] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:31.040243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:31.166014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39e6a8-a563-4077-a69e-47399b7d1320 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:31.171076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:31.186642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e20fc-f7d6-4591-b842-75ea15c7eafd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:31.211947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:31.212828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:31.213127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:31.213407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.214314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:31.214938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.216633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:31.224217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:31.225138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:31.225499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:31.225923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.234083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.234284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 355/698] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:31.235745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:31.245765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:31.254273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:31.255160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:31.260648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 344/699] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:31.261908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:31.306622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:31.372554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0b6ce-f187-4cb2-90c3-e2459be58249 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:31.406396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf131f-3d83-4eeb-8e5c-6b8623310fc5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:31.410258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:31.410581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:31.410960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:31.411310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.412189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:31.412570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.416802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:31.428664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 345/700] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:31.434756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:31.439348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:31.451681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:31.452861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:31.453329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:31.453780 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.461749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.463647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:31.479090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:31.480076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:31.481052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:31.484218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 356/701] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:25:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:31.490151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:31.601915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:31.610652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c5384-b7b2-4268-aa69-face247a6e0b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:31.651399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:31.652384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:31.652868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:31.653458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.655239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:31.655816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.657486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:31.663789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65b205-effe-4cb9-889e-c2529844158d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:31.679326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 357/702] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:25:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:31.687274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:31.736363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e1e08-df26-466b-aa59-ed29b0a90310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:31.736976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e1e08-df26-466b-aa59-ed29b0a90310 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:31.737262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e1e08-df26-466b-aa59-ed29b0a90310 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:31.737541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e1e08-df26-466b-aa59-ed29b0a90310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.738383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e1e08-df26-466b-aa59-ed29b0a90310 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=642a42337be14f10b410903df57f7527) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:31.738703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e1e08-df26-466b-aa59-ed29b0a90310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.740179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e1e08-df26-466b-aa59-ed29b0a90310 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:31.761566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 346/703] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:31 2022] GET /identity/v3/role_assignments?user.id=642a42337be14f10b410903df57f7527&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:31.836400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533532-5827-4622-b9c6-60d675bcc6fd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:31.880562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:31.881319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:31.881607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:31.881888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:31.920101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:31.992905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:31.994319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.069395 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067fbb9-6b4f-4556-a76e-cd9ba7f5cd00 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.069730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067fbb9-6b4f-4556-a76e-cd9ba7f5cd00 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.069941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067fbb9-6b4f-4556-a76e-cd9ba7f5cd00 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.071681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 347/704] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:32.077880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542aa08-fade-48a1-ab5b-1b920975eaaa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.078220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542aa08-fade-48a1-ab5b-1b920975eaaa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.078501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542aa08-fade-48a1-ab5b-1b920975eaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.100705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8542aa08-fade-48a1-ab5b-1b920975eaaa None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:32.101944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8542aa08-fade-48a1-ab5b-1b920975eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:32.132747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 358/705] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:32.138956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:32.184811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 348/706] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:32.184811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:32.210129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542aa08-fade-48a1-ab5b-1b920975eaaa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:32.214514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:32.276187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061f597-8a29-4339-bc61-21be1efc86cf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:32.315452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:32.316512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.317042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.317522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.327606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542aa08-fade-48a1-ab5b-1b920975eaaa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:32.329953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:32.331632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:32.351200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:32.352156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:32.353152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:32.353152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:32.353152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:32.353152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:32.355483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 359/707] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:32.361347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:32.366666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5c9de-c01f-40f9-8d2f-4a828bd08e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:32.367334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5c9de-c01f-40f9-8d2f-4a828bd08e96 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.367612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5c9de-c01f-40f9-8d2f-4a828bd08e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.367885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5c9de-c01f-40f9-8d2f-4a828bd08e96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.383770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5c9de-c01f-40f9-8d2f-4a828bd08e96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:32.385298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5c9de-c01f-40f9-8d2f-4a828bd08e96 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-dee5c9de-c01f-40f9-8d2f-4a828bd08e96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:32.406980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:32.408016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:32.409054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:32.409054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:32.409054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:32.409054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:32.412900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 349/708] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:32.414331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc141d91-f17d-419f-9beb-c119b76bf5a5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.414625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc141d91-f17d-419f-9beb-c119b76bf5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.414961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc141d91-f17d-419f-9beb-c119b76bf5a5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.417429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 350/709] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:32.423240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc141d91-f17d-419f-9beb-c119b76bf5a5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:32.485178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:32.502538 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f051f-cbf8-4f5b-b873-901d7b887f78 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:32.506022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:32.535661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc141d91-f17d-419f-9beb-c119b76bf5a5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:32.546743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ec592-0ebc-413b-b8b2-09b1f26aef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:32.547517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ec592-0ebc-413b-b8b2-09b1f26aef2a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.547873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ec592-0ebc-413b-b8b2-09b1f26aef2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.548222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ec592-0ebc-413b-b8b2-09b1f26aef2a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.549098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ec592-0ebc-413b-b8b2-09b1f26aef2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:32.549477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ec592-0ebc-413b-b8b2-09b1f26aef2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:32.551070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ec592-0ebc-413b-b8b2-09b1f26aef2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.564789 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 360/710] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:32.566553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47752-115f-47d0-99cc-79339827bd36 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.566919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47752-115f-47d0-99cc-79339827bd36 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.567268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47752-115f-47d0-99cc-79339827bd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.578208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:32.578955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.579244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.579526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.580359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:32.580707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:32.583100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.586993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-72b47752-115f-47d0-99cc-79339827bd36 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:32.588336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-72b47752-115f-47d0-99cc-79339827bd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:32.597637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 351/711] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:32.598513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:32.684875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 361/712] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:32.686447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47752-115f-47d0-99cc-79339827bd36 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:32.734552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5923052-46ac-43c5-b23e-c4b11543c77c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:32.746187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:32.775272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:32.776541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.777038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.777485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.785819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:32.787834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:32.805229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:32.806103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 352/713] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:32.807639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:32.829664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:32.840408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47752-115f-47d0-99cc-79339827bd36 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:32.887275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8abd4-696a-45da-896d-79421db4cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:32.888105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8abd4-696a-45da-896d-79421db4cefb None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.888489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8abd4-696a-45da-896d-79421db4cefb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.888892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8abd4-696a-45da-896d-79421db4cefb None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.890310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8abd4-696a-45da-896d-79421db4cefb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:32.891864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8abd4-696a-45da-896d-79421db4cefb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.935661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6f6f4-71e6-45ba-b7fb-8bcbe9e93192 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:32.972182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:32.973022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:32.973307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:32.973665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:32.974338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:32.975732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:32.989682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:32.990683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:32.991555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 353/714] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:32 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:32.992477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:33.011439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 362/715] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:25:32 2022] POST /identity/v3/users => generated 254 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:33.022428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4d253-3bc5-43d1-82c2-b7b488d36ab9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:33.022719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4d253-3bc5-43d1-82c2-b7b488d36ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:33.023022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4d253-3bc5-43d1-82c2-b7b488d36ab9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:33.024549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 363/716] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:33.025183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4d253-3bc5-43d1-82c2-b7b488d36ab9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:33.053347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:33.157555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18baed2f-928e-480b-87ec-77896ec37a7c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:33.166323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4d253-3bc5-43d1-82c2-b7b488d36ab9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:33.200255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a8089-e4fc-43b1-827b-c13badc443ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:33.201271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a8089-e4fc-43b1-827b-c13badc443ff None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:33.201854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a8089-e4fc-43b1-827b-c13badc443ff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:33.202475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a8089-e4fc-43b1-827b-c13badc443ff None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:33.203355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a8089-e4fc-43b1-827b-c13badc443ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:33.203923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a8089-e4fc-43b1-827b-c13badc443ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:33.205436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a8089-e4fc-43b1-827b-c13badc443ff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:33.216844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:33.218167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:33.218456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:33.218736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:33.219455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] RBAC: Adding query filter params (name=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:33.219765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:33.222264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 354/717] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:33.222395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:33.222474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee968d-0948-441c-8984-071cec026476 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:33.222474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee968d-0948-441c-8984-071cec026476 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:33.222474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee968d-0948-441c-8984-071cec026476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:33.227898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 364/718] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:25:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:33.228740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:33.240887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-08ee968d-0948-441c-8984-071cec026476 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:33.242416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-08ee968d-0948-441c-8984-071cec026476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:33.254295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:33.343979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 355/719] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:33.345744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee968d-0948-441c-8984-071cec026476 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:33.373225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75938f0c-254d-43fb-9dff-a67e973b7944 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:33.375383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:33.428808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:33.428808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:33.428808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:33.428808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:33.429273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=ec899ae414744dde9ec9b76aac1ce9d8) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:33.429273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:33.430010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:33.436576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 365/720] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:33 2022] GET /identity/v3/role_assignments?user.id=ec899ae414744dde9ec9b76aac1ce9d8&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:33.437706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:33.515803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee968d-0948-441c-8984-071cec026476 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:33.567959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6de30-e33d-4fef-a9c7-7ba6ea0fbd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:33.568695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6de30-e33d-4fef-a9c7-7ba6ea0fbd55 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:33.569081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6de30-e33d-4fef-a9c7-7ba6ea0fbd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:33.569402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6de30-e33d-4fef-a9c7-7ba6ea0fbd55 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:33.570530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6de30-e33d-4fef-a9c7-7ba6ea0fbd55 None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:33.570843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6de30-e33d-4fef-a9c7-7ba6ea0fbd55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:33.572257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6de30-e33d-4fef-a9c7-7ba6ea0fbd55 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:33.580609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 356/721] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:25:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:33.646109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:33.649859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036f26-479d-49e4-9b1c-92dc4a66c71d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:33.699643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a33af-35d2-4e4c-bc18-fc8f1c2d4292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:33.700853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a33af-35d2-4e4c-bc18-fc8f1c2d4292 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:33.701363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a33af-35d2-4e4c-bc18-fc8f1c2d4292 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:33.702156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a33af-35d2-4e4c-bc18-fc8f1c2d4292 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:33.703268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a33af-35d2-4e4c-bc18-fc8f1c2d4292 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:33.704993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a33af-35d2-4e4c-bc18-fc8f1c2d4292 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:33.862144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 366/722] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:25:33 2022] POST /identity/v3/services => generated 251 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:34.390225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564069a-42d3-4dea-8c54-97926e3d50fe None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:34.390746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564069a-42d3-4dea-8c54-97926e3d50fe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:34.390909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564069a-42d3-4dea-8c54-97926e3d50fe None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:34.392526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 357/723] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:34.398096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd014cf9-b0c2-4690-b770-83b0e2dd1aba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:34.398908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd014cf9-b0c2-4690-b770-83b0e2dd1aba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:34.399366 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd014cf9-b0c2-4690-b770-83b0e2dd1aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:34.418756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cd014cf9-b0c2-4690-b770-83b0e2dd1aba None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:34.420286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd014cf9-b0c2-4690-b770-83b0e2dd1aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:34.526849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 367/724] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:34.572500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564069a-42d3-4dea-8c54-97926e3d50fe None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:34.683236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:34.697085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564069a-42d3-4dea-8c54-97926e3d50fe None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:34.733283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:34.734324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:34.734606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:34.734883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:34.746288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:34.756299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:34.757346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:34.758596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 358/725] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:34.764087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd014cf9-b0c2-4690-b770-83b0e2dd1aba None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:34.860225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:34.899972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd014cf9-b0c2-4690-b770-83b0e2dd1aba None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:34.935557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:34.936682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:34.936995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:34.937272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:34.938031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:34.938339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:34.939746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:34.949188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 368/726] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:34.955145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:35.131720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110ff4-be80-4b52-a2c9-5305dae13b40 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:35.167830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:35.168678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:35.169025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:35.169307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:35.181617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:35.183141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:35.192403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:35.202947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:35.203857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:35.204818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:35.204818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:35.204818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:35.204818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:35.204818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 359/727] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:35.210157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:35.335898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f873-4c02-4bcf-bb4e-c85fd53511a7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:35.371394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:35.384272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fca62a-a485-49cb-a6b1-edcdc9119983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:35.384969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fca62a-a485-49cb-a6b1-edcdc9119983 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:35.385251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fca62a-a485-49cb-a6b1-edcdc9119983 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:35.385524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fca62a-a485-49cb-a6b1-edcdc9119983 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:35.386308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fca62a-a485-49cb-a6b1-edcdc9119983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:35.386684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fca62a-a485-49cb-a6b1-edcdc9119983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:35.388048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fca62a-a485-49cb-a6b1-edcdc9119983 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:35.403926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 369/728] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:35.410375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:35.463841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f7a04-8155-41db-86a5-279841efb98d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:35.464141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f7a04-8155-41db-86a5-279841efb98d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:35.464325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f7a04-8155-41db-86a5-279841efb98d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:35.467620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 370/729] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:35.472852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756235cc-f0be-4cf9-a50c-69c422a3eced None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:35.473140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756235cc-f0be-4cf9-a50c-69c422a3eced None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:35.473429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756235cc-f0be-4cf9-a50c-69c422a3eced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:35.491534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-756235cc-f0be-4cf9-a50c-69c422a3eced None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:35.492734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-756235cc-f0be-4cf9-a50c-69c422a3eced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:35.532746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786286a-43e4-4e14-94bf-e34f9a37010b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:35.570029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:35.570945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:35.571231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:35.571534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:35.580389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:35.581890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:35.597548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:35.598568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:35.599529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 360/730] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:35.604682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:35.636149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 371/731] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:25:35.674956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:35.682239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756235cc-f0be-4cf9-a50c-69c422a3eced None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:35.727820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72a5d-cad2-4b7e-ba21-b65e9b12ca5c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:35.777527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:35.780276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:35.780767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:35.781072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:35.781935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:35.782246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:35.783718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:35.828820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 361/732] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 18 10:25:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:35.831445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:35.901823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756235cc-f0be-4cf9-a50c-69c422a3eced None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:35.921196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:35.948989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:35.950991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:35.951381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:35.951790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:35.974771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:35.994255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:35.995232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:35.996124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:35.998282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 372/733] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:36.002445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.029933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444963cc-c6ee-4af3-8d19-91812033668c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.051043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:36.065336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19117a-b995-442e-a46e-e70e9b5fcbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.065993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19117a-b995-442e-a46e-e70e9b5fcbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.066282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19117a-b995-442e-a46e-e70e9b5fcbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.066560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19117a-b995-442e-a46e-e70e9b5fcbd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.153047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19117a-b995-442e-a46e-e70e9b5fcbd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:36.155046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19117a-b995-442e-a46e-e70e9b5fcbd8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:36.165170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac2dc-0c6b-4158-8a08-5f2f2bd99267 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.203190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888a5f1-ba01-4e13-bd47-4c90870d3e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.203865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a5f1-ba01-4e13-bd47-4c90870d3e5f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.204146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a5f1-ba01-4e13-bd47-4c90870d3e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.204426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a5f1-ba01-4e13-bd47-4c90870d3e5f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.205216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a5f1-ba01-4e13-bd47-4c90870d3e5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:36.205526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a5f1-ba01-4e13-bd47-4c90870d3e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:36.206895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a5f1-ba01-4e13-bd47-4c90870d3e5f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:36.215681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 373/734] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:36.217637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d868b-deeb-47af-aa2b-67034f097d69 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.218086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d868b-deeb-47af-aa2b-67034f097d69 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.218576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d868b-deeb-47af-aa2b-67034f097d69 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.223884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 374/735] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:36.225167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6c95-cf83-4e07-b56a-4c84574e88e5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.225458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6c95-cf83-4e07-b56a-4c84574e88e5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.225735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6c95-cf83-4e07-b56a-4c84574e88e5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.226982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 375/736] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:36.228994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6c95-cf83-4e07-b56a-4c84574e88e5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.274303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:36.280951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 362/737] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:35 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:25:36.282556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a324-821c-4a8a-b107-7d673640fd26 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.282913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a324-821c-4a8a-b107-7d673640fd26 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.283201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a324-821c-4a8a-b107-7d673640fd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.303495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e3a8a324-821c-4a8a-b107-7d673640fd26 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:36.304706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e3a8a324-821c-4a8a-b107-7d673640fd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:36.323517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:36.344381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6c95-cf83-4e07-b56a-4c84574e88e5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.379933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.380748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.381140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.381418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.393745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:36.395202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:36.404915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 363/738] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:36.404915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44b578-e8bd-42e1-a5ee-cb5b9df75858 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.404915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44b578-e8bd-42e1-a5ee-cb5b9df75858 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.404915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44b578-e8bd-42e1-a5ee-cb5b9df75858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:36.413518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:36.414842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:36.415731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:36.415731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:36.415731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:36.415731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:36.415731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 376/739] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:36.416013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.429014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ad44b578-e8bd-42e1-a5ee-cb5b9df75858 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:36.430346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad44b578-e8bd-42e1-a5ee-cb5b9df75858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:36.504831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 364/740] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:25:36.504831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44b578-e8bd-42e1-a5ee-cb5b9df75858 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.529354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:36.540130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346379ad-4cb2-4145-8e24-ecd336bdba05 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.577316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.578119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.578406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.578683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.590497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:36.592069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:36.600146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:36.611069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:36.614257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:36.615190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:36.615190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:36.615190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:36.615190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:36.615190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 377/741] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:36.615190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.632292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44b578-e8bd-42e1-a5ee-cb5b9df75858 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.672428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.673507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.674079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.674563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.675553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:36.676046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:36.677950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:36.692839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 365/742] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:36.692839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.727614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ea0ff-f422-4954-8f8a-f2d663062a0a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.762282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.763177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.763462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.764363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.774831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:36.783564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:36.784716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 378/743] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:36.786036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.788899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:36.803873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d44560-259d-464f-9a42-e67da468e088 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.839083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f134d-793f-4efd-9152-2c75f33a31d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.839748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f134d-793f-4efd-9152-2c75f33a31d4 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.840035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f134d-793f-4efd-9152-2c75f33a31d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.840305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f134d-793f-4efd-9152-2c75f33a31d4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.841069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f134d-793f-4efd-9152-2c75f33a31d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:36.843168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f134d-793f-4efd-9152-2c75f33a31d4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:36.886988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e4c5-2b91-49ac-b296-35fa1726372e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:36.921096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:36.921763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.922049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.922323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.923047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:36.923356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:36.924735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:36.937029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:36.937296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 379/744] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:36.938092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:36.947902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 366/745] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:25:36 2022] POST /identity/v3/services => generated 240 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:36.949367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310438b0-fe2d-4a2f-9ca0-7dbc0b26074a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:36.949606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310438b0-fe2d-4a2f-9ca0-7dbc0b26074a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:36.949886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310438b0-fe2d-4a2f-9ca0-7dbc0b26074a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:36.952195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 367/746] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:36.952984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310438b0-fe2d-4a2f-9ca0-7dbc0b26074a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.023772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:37.067583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c2a73-1048-4162-b22b-f5d091ce39ab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.083142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310438b0-fe2d-4a2f-9ca0-7dbc0b26074a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.108668 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.109528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.109857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.110100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.117894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.120107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.126454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39f5ef-8c5d-480e-be7e-ee1e5b5c07a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.126906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39f5ef-8c5d-480e-be7e-ee1e5b5c07a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.127265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39f5ef-8c5d-480e-be7e-ee1e5b5c07a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.127544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39f5ef-8c5d-480e-be7e-ee1e5b5c07a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.128488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39f5ef-8c5d-480e-be7e-ee1e5b5c07a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:37.128824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39f5ef-8c5d-480e-be7e-ee1e5b5c07a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.130282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39f5ef-8c5d-480e-be7e-ee1e5b5c07a6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:37.137773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:37.139426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 380/747] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:37.141422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.142379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 368/748] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:37.142379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636569d-1401-4f5b-bc36-711e728db08d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.142516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636569d-1401-4f5b-bc36-711e728db08d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.142767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636569d-1401-4f5b-bc36-711e728db08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.163310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5636569d-1401-4f5b-bc36-711e728db08d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:37.164505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5636569d-1401-4f5b-bc36-711e728db08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:37.225236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:37.256925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 369/749] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:37.257849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636569d-1401-4f5b-bc36-711e728db08d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.265563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be142915-5bf5-43d2-9480-10439e1212f1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.293273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:37.305119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.305922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.306201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.306480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.313879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.315917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:37.331844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:37.332754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:37.333728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 381/750] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:37.334264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.381869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636569d-1401-4f5b-bc36-711e728db08d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.421208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.421925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.422218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.422496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.423342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:37.423653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.425219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.444527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 370/751] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:37.445149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.452466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89658d25-306f-46dd-b68c-f8a6b3c16b2c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.490839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.491635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.491924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.492208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.494291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:37.504542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.506046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:37.525317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:37.526909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:37.528666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:37.528666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:37.528666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:37.528666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:37.528666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 382/752] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:37.528666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.561521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9bc66-4e92-4c68-906d-807813752681 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.566469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:37.602349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.603085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.603366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.603644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.604312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.605725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:37.618959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:37.620173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:37.621082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 371/753] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:37.621935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.653139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dd3f7-6790-405f-9367-3d3d9d86f435 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.692403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.693195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.693504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.693783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.694571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:37.694879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.696425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.715457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 383/754] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:37.715457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.755183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec17651-a518-43b7-bd32-1e1425da436f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.765795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:37.791746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.792387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.792665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.792966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.824072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:37.831214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6486c46-180f-4f75-bd5b-ed11b23c9232 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:37.867625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:37.868290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:37.868573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:37.868876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:37.869648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:37.869955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.871321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.873246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:37.874770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:37.887935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 384/755] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:37.887935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.989723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 372/756] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:25:37.990805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:37.999992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bdc0f-aa1c-4364-b90b-a1cf71c43bf4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.003684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.038877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.039552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.039838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.040109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.040844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:38.041156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.042575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.047810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 385/757] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:38.048715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.071436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.103030 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ee125-5bbf-4301-8014-a83f93e6ff5d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.143988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.144565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.144873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.145153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.145995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=642a42337be14f10b410903df57f7527) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:38.146315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.147790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.154299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 373/758] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:37 2022] GET /identity/v3/role_assignments?user.id=642a42337be14f10b410903df57f7527&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:38.155147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.182990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1137583-b283-48cb-b74f-e971cdd9f80f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.227472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.228104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.228389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.228660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.237640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.239229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.253373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:38.255887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:38.256893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:38.257887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:38.261155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 386/759] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:38.262136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.300610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8157-94b5-4c52-b9e4-1538ca08d5da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.348002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.349260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.349730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.350640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.353773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.365775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.368015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.377317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619ef1-2773-42f7-a9aa-49cf05a6797e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:38.396235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:38.397388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:38.398527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:38.398527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:38.398527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:38.398527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:38.405716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 374/760] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:38.405716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.438490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.439742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.440184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.440619 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.442214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:38.442690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.444832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.457626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 387/761] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:38.458232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.534689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a74ac3-29d5-48d0-8792-f41105c0ad53 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.570024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.581814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.582618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.582966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.583438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.583657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e64a7-ee2f-4650-93dc-f1140cfbb389 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.584485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:38.584924 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.586473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.611257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 375/762] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:38.612391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.630794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.631992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.632435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.633469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.634529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:38.635102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.636969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.660375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 388/763] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:38.660375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.666251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.750554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b10697-290d-487c-88c0-8483bb809acf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.792652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b406f-aac7-4cd6-9ad4-b794a1b79092 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.796589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.797982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.798433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.798874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.799851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] RBAC: Adding query filter params (interface=public, service_id=095ed85b52d3488dba6ff2315053718d, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:38.800318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.801953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.821517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 376/764] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/endpoints?service_id=095ed85b52d3488dba6ff2315053718d&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:38.822242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.839143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:38.840003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:38.840550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:38.840550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:38.860342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:38.861899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:38.881914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:38.882893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:38.883815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:38.883815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:38.883815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:38.883815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:38.887738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 389/765] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:38.888503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.888665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:38.954247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:38.961799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51946487-a2db-4671-acfb-5909a2e01f33 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:38.999792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.000978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.001542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.002133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.005386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a246c-e57f-4b3a-a132-490d0f17678c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.010077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.011818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:39.029420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:39.030378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:39.031414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:39.038645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 377/766] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:39.039877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.041685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.042437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.042881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.043246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.044127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.046446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.051995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 390/767] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:25:38 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:39.052895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.181190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2823fb-4441-452e-93ea-665a9110ae2c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.182460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff3c20-52e3-41ab-aa5b-b271593b062d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.196388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:39.203671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:39.228150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a18811-064c-4ce6-aa95-7f2a3b3b9a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.228747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a18811-064c-4ce6-aa95-7f2a3b3b9a0a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.229146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a18811-064c-4ce6-aa95-7f2a3b3b9a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.229448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a18811-064c-4ce6-aa95-7f2a3b3b9a0a None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.230326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a18811-064c-4ce6-aa95-7f2a3b3b9a0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:39.230640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a18811-064c-4ce6-aa95-7f2a3b3b9a0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.232087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a18811-064c-4ce6-aa95-7f2a3b3b9a0a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.236001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.236702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.238193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.240672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.240672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:39.240672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.241916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.246352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 391/768] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:39.248418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fa3f8-1545-43d9-912c-db3399d6ff13 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.249032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fa3f8-1545-43d9-912c-db3399d6ff13 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.249543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fa3f8-1545-43d9-912c-db3399d6ff13 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.253033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 392/769] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:39.256201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 378/770] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:39.259503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79b071-25ba-40e1-a5a6-c6ae16f8299b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.260046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79b071-25ba-40e1-a5a6-c6ae16f8299b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.260177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.260354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79b071-25ba-40e1-a5a6-c6ae16f8299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.278524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5d79b071-25ba-40e1-a5a6-c6ae16f8299b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:39.280597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5d79b071-25ba-40e1-a5a6-c6ae16f8299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:39.369691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 393/771] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:39.370566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79b071-25ba-40e1-a5a6-c6ae16f8299b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.395953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a1e75-c89a-453d-9aa8-d8e6eb695126 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.444090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.445044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.445734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.446020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.447172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=41cc665cbec74e06bfa7acf97f968e52) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:39.447485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.449865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.458510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 379/772] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:39 2022] GET /identity/v3/role_assignments?user.id=41cc665cbec74e06bfa7acf97f968e52&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:39.459513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.493765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79b071-25ba-40e1-a5a6-c6ae16f8299b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.506889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:39.508218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:39.538301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.539285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.539675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.540054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.547810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.549468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:39.567880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:39.570775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:39.572923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 394/773] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:39.576914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.597578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa431b-994b-4ed9-806e-b9199c106c50 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.641449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.642269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.642554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.642831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.643498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.647075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:39.660677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:39.663119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:39.665138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 380/774] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:39 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:39.674100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.829785 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a3284-097e-47b9-a323-98c6c47c158e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.886322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:39.887028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:39.887309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:39.887580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:39.892570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:39.892918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:39.894400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:39.925090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 395/775] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:39.931063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:39.943487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:39.955839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b59b6-91ce-4f87-a1ac-fea7da54d755 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:39.997157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:40.045845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:40.046521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.046804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.047075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.047833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:40.048138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:40.049645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:40.056009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 381/776] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:25:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:40.062049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:40.117656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843d93c-568a-44de-8104-4452f05e4c1b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:40.162516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:40.163495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.163777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.164051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/ec899ae414744dde9ec9b76aac1ce9d8/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.171041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=ec899ae414744dde9ec9b76aac1ce9d8, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:40.172649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:40.229719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef238a-bc5e-487d-aeed-22c06cd0dfb1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:40.268612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:40.269341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.269791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.270071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.270869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:40.271188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:40.272627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:40.279630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 382/777] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:25:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:40.290191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:40.307402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 396/778] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:39 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/ec899ae414744dde9ec9b76aac1ce9d8/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:25:40.334301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:40.462997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4745-606d-40e0-88b4-0952d4297fb1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:40.505945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf8962-36ca-45e1-bce7-e3542287ea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:40.506662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf8962-36ca-45e1-bce7-e3542287ea95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.506950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf8962-36ca-45e1-bce7-e3542287ea95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.507225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf8962-36ca-45e1-bce7-e3542287ea95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.508020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf8962-36ca-45e1-bce7-e3542287ea95 None admin] RBAC: Adding query filter params (interface=public, service_id=d31eb034d35e481fa2beb8cd97ea1bc1, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:40.508333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf8962-36ca-45e1-bce7-e3542287ea95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:40.509784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf8962-36ca-45e1-bce7-e3542287ea95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:40.516590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 383/779] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:25:40 2022] GET /identity/v3/endpoints?service_id=d31eb034d35e481fa2beb8cd97ea1bc1&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:40.522350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:40.532932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b026f9-cfd1-4805-b393-765353c0f585 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.533232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b026f9-cfd1-4805-b393-765353c0f585 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.533494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b026f9-cfd1-4805-b393-765353c0f585 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.535177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 384/780] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:40.541719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d29f7-9f86-48ed-a710-df24c8bfd274 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.542722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d29f7-9f86-48ed-a710-df24c8bfd274 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.544385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d29f7-9f86-48ed-a710-df24c8bfd274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.584198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ac5d29f7-9f86-48ed-a710-df24c8bfd274 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:40.585536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ac5d29f7-9f86-48ed-a710-df24c8bfd274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:40.653405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:40.674450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f774c-4ec7-49e8-a44d-0dd33b2641d5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:40.710532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 385/781] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:25:40.718800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:40.719489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.719779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.720080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.720832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:40.722256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:40.734259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 397/782] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:25:40 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:40.736159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d29f7-9f86-48ed-a710-df24c8bfd274 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:40.769444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:40.892443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d29f7-9f86-48ed-a710-df24c8bfd274 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:40.937257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:40.938157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:40.938447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:40.938759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:40.955990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:40.964010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:40.965124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:40.966046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:40.969767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 386/783] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:40.975418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:41.091043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566bb53-234c-4e91-b4a5-203d438071f7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:41.138965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4995f2-36a5-4fa6-b1b0-f061fafaf3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:41.139690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4995f2-36a5-4fa6-b1b0-f061fafaf3aa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:41.139981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4995f2-36a5-4fa6-b1b0-f061fafaf3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:41.140262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4995f2-36a5-4fa6-b1b0-f061fafaf3aa None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:41.141050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4995f2-36a5-4fa6-b1b0-f061fafaf3aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:41.141364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4995f2-36a5-4fa6-b1b0-f061fafaf3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:41.142792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4995f2-36a5-4fa6-b1b0-f061fafaf3aa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:41.157014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 398/784] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:41.162488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:41.316715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:41.347797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91f1d4-9054-4c7e-9d95-7266484cea88 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:41.388683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:41.389524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:41.389806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:41.390081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:41.401565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:41.402972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:41.421660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:41.422690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:41.423596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:41.423596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:41.423596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:41.423596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:41.423596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 387/785] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:41.427216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6d060-935b-49f2-b3c7-463df692d0d2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:41.427699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6d060-935b-49f2-b3c7-463df692d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:41.427912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6d060-935b-49f2-b3c7-463df692d0d2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:41.428658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:41.432883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 399/786] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:41.438194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c0acf-6890-487b-94e2-132a8d587ddf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:41.438489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c0acf-6890-487b-94e2-132a8d587ddf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:41.438790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c0acf-6890-487b-94e2-132a8d587ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:41.466159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-515c0acf-6890-487b-94e2-132a8d587ddf None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:41.467422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-515c0acf-6890-487b-94e2-132a8d587ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:41.488749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:41.589645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 400/787] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:41.591907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9be33e-87b9-4cf0-a7cb-bea033ea135a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:41.639378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:41.641589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c0acf-6890-487b-94e2-132a8d587ddf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:41.660936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:41.661538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:41.661825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:41.662102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:41.662841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:41.663154 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:41.664554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:41.687962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 388/788] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:41.693723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:41.845116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c0acf-6890-487b-94e2-132a8d587ddf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:41.919073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a26b7c-b1c1-4bd8-a346-e380d339c1e7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:41.923342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:41.924414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:41.924948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:41.925445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:41.927254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:41.929147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:41.960279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:41.961386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:41.962215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 401/789] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Jul 18 10:25:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:41.968485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:41.976126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb6434-594f-4e34-8428-7ab4f9f77f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:41.977967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb6434-594f-4e34-8428-7ab4f9f77f2c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:41.978580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb6434-594f-4e34-8428-7ab4f9f77f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:41.979101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb6434-594f-4e34-8428-7ab4f9f77f2c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:41.992383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb6434-594f-4e34-8428-7ab4f9f77f2c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:41.994812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:41.997263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb6434-594f-4e34-8428-7ab4f9f77f2c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-91bb6434-594f-4e34-8428-7ab4f9f77f2c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:42.024754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:42.026132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:42.027103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:42.033449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 389/790] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:42.042338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713777d-977e-4550-901c-2043df13bcd8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.042634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713777d-977e-4550-901c-2043df13bcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.042959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713777d-977e-4550-901c-2043df13bcd8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.044324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 390/791] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:42.046089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713777d-977e-4550-901c-2043df13bcd8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.109858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:42.124256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83f253-be3a-4709-bef8-b75eb294d171 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.161358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff234ec-692b-4636-a2f5-b30ea16568d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.162152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff234ec-692b-4636-a2f5-b30ea16568d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.162536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff234ec-692b-4636-a2f5-b30ea16568d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.162906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff234ec-692b-4636-a2f5-b30ea16568d9 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.163694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff234ec-692b-4636-a2f5-b30ea16568d9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:42.164081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff234ec-692b-4636-a2f5-b30ea16568d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:42.165582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff234ec-692b-4636-a2f5-b30ea16568d9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.174983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713777d-977e-4550-901c-2043df13bcd8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.175694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 402/792] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 18 10:25:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:42.177362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1edc64-feed-4073-8e18-b557e6114748 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.177721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1edc64-feed-4073-8e18-b557e6114748 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.178073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1edc64-feed-4073-8e18-b557e6114748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.200624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-de1edc64-feed-4073-8e18-b557e6114748 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:42.202758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-de1edc64-feed-4073-8e18-b557e6114748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:42.217031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.217871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.218191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.218554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.219403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:42.219713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:42.221285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.241443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 391/793] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 18 10:25:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:42.241844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.286221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 403/794] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:42.287295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1edc64-feed-4073-8e18-b557e6114748 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.295567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:42.355872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57117c-268d-4668-a9f7-3424ad871e5f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.364934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:42.395092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.395694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.395975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.396253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.397309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:42.398155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:42.400312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.411672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 392/795] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 18 10:25:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:42.411672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.419067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1edc64-feed-4073-8e18-b557e6114748 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.469360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967060e9-1caf-441e-828a-e236904af908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.470408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967060e9-1caf-441e-828a-e236904af908 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.470880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967060e9-1caf-441e-828a-e236904af908 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.471329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967060e9-1caf-441e-828a-e236904af908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.487354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967060e9-1caf-441e-828a-e236904af908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:42.489158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967060e9-1caf-441e-828a-e236904af908 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-967060e9-1caf-441e-828a-e236904af908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:42.512618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:42.514566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:42.515503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:42.515503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:42.515503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:42.515503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:42.520638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 404/796] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:42.521664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967060e9-1caf-441e-828a-e236904af908 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.527245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1466e-a6bd-42e0-bcd0-d894dfa4a33e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.563821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.564982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.565468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.565923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.577620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:42.583521 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:42.585976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:42.586916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:42.587776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:42.595658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 393/797] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:42.603028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.629127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:42.634819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967060e9-1caf-441e-828a-e236904af908 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.673183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c0b7f-2e4f-4893-9fdc-ccf5325b3646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.673856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c0b7f-2e4f-4893-9fdc-ccf5325b3646 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.674143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c0b7f-2e4f-4893-9fdc-ccf5325b3646 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.674421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c0b7f-2e4f-4893-9fdc-ccf5325b3646 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.675266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c0b7f-2e4f-4893-9fdc-ccf5325b3646 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:42.676614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c0b7f-2e4f-4893-9fdc-ccf5325b3646 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.704389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b9594-fd3a-4ab9-81ac-aa9bcef85b41 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.738629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.739251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.739532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.739836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.740566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:42.740900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:42.742282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.756162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 394/798] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:42.757138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.799498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 405/799] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:25:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:42.801176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0ca8-1099-4904-a671-e547fa0606e6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.801650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0ca8-1099-4904-a671-e547fa0606e6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.802100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0ca8-1099-4904-a671-e547fa0606e6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.803902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 406/800] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:42.804916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa0ca8-1099-4904-a671-e547fa0606e6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:42.908850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:42.925097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ebdc8-c5d6-40d2-a888-ed0bea547767 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.970640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa0ca8-1099-4904-a671-e547fa0606e6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:42.971435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28492d49-9a8d-452a-82d1-55240b50b2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:42.972164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28492d49-9a8d-452a-82d1-55240b50b2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.972647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28492d49-9a8d-452a-82d1-55240b50b2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.972981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28492d49-9a8d-452a-82d1-55240b50b2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:42.975200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28492d49-9a8d-452a-82d1-55240b50b2a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:42.975200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28492d49-9a8d-452a-82d1-55240b50b2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:42.976667 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28492d49-9a8d-452a-82d1-55240b50b2a7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:42.978610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:42.988501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 395/801] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:42.989785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85e1f1-56c5-4671-b7c8-b7027e3e429d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:42.990021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85e1f1-56c5-4671-b7c8-b7027e3e429d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:42.990315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85e1f1-56c5-4671-b7c8-b7027e3e429d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.007795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dc85e1f1-56c5-4671-b7c8-b7027e3e429d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:43.009012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dc85e1f1-56c5-4671-b7c8-b7027e3e429d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:43.017545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.018754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.019743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.020219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.029289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.032976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:43.047450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:43.048418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:43.049398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 407/802] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:43.050432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.100466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 396/803] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:43.107493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85e1f1-56c5-4671-b7c8-b7027e3e429d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.174759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3467683-2d39-4083-aa0d-678f46b16323 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.211722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:43.213514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.214530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.214997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.215454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.228182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.230072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.232559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85e1f1-56c5-4671-b7c8-b7027e3e429d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:43.249654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:43.250602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:43.251492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:43.251492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:43.251492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:43.251492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:43.253977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 408/804] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:43.258642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.271609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c5d8-e13f-41c2-bcaf-bab6e876b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.271981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c5d8-e13f-41c2-bcaf-bab6e876b52a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.272264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c5d8-e13f-41c2-bcaf-bab6e876b52a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.272538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c5d8-e13f-41c2-bcaf-bab6e876b52a None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.273371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921c5d8-e13f-41c2-bcaf-bab6e876b52a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:43.273680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921c5d8-e13f-41c2-bcaf-bab6e876b52a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.275121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921c5d8-e13f-41c2-bcaf-bab6e876b52a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.284076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:43.287897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 397/805] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:43.289017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942090-f6ab-4fe0-a9e6-71093021984c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.289309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942090-f6ab-4fe0-a9e6-71093021984c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.289584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942090-f6ab-4fe0-a9e6-71093021984c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.290904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 398/806] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:43.291495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90942090-f6ab-4fe0-a9e6-71093021984c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.390238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff784f3-8d10-4520-aa09-453df8c1b8e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.406960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90942090-f6ab-4fe0-a9e6-71093021984c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.426658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.427463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.427762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.428034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.439406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:43.445927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33747b65-c598-4b28-a045-f83dee3e8a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.446599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33747b65-c598-4b28-a045-f83dee3e8a4e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.446878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33747b65-c598-4b28-a045-f83dee3e8a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.447153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33747b65-c598-4b28-a045-f83dee3e8a4e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:43.447458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:43.448392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:43.454299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:43.454957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33747b65-c598-4b28-a045-f83dee3e8a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:43.454957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33747b65-c598-4b28-a045-f83dee3e8a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.455104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 409/807] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:43.455179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33747b65-c598-4b28-a045-f83dee3e8a4e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.455247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.469238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 399/808] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:43.475157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615cd5a-93dc-4e9e-8052-44bebf652f48 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.475451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615cd5a-93dc-4e9e-8052-44bebf652f48 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.475737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615cd5a-93dc-4e9e-8052-44bebf652f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.480751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:43.493204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2615cd5a-93dc-4e9e-8052-44bebf652f48 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:43.495728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2615cd5a-93dc-4e9e-8052-44bebf652f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:43.550818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:43.573469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d523607-ea53-4cf0-bf15-abc30c648d68 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.574659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 400/809] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:43.575592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615cd5a-93dc-4e9e-8052-44bebf652f48 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.622356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.624220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.624508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.624847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/642a42337be14f10b410903df57f7527/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.627033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=642a42337be14f10b410903df57f7527, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.628501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.685752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615cd5a-93dc-4e9e-8052-44bebf652f48 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.712816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 410/810] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:43 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/642a42337be14f10b410903df57f7527/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:43.712816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.722289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:43.724174 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.724896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.725228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.725502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.726267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:43.726573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.728088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.738616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 401/811] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:43.744478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.871507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e863ebe-21e3-408b-a397-cba567338170 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.878565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ea6b3-670d-40d4-8a04-da7df6857344 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:43.917356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.918635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.919124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.919830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.920034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:43.920729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:43.921046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:43.921325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:43.922057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.924672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.930948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:43.932456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:43.934460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:43.940635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:43.942231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:25:43.943285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:43.946009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 402/812] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:43.947065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:43.957034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:43.958233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:43.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:43.961362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 411/813] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:43.962253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.017035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:44.064025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be484b-2ab4-4903-bbc2-ac1ff4f1492e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.077973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680246c-6465-4044-8d35-8962ddbc79f6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.102127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.103197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.103636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.104066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.116808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.117523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.118350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.118742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.118968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.119215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.120063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] RBAC: Adding query filter params (name=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:44.120474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.122177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.128393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 412/814] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:44.129779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:44.139830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:44.141000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:44.142206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:44.142206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:44.142206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:44.142206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:44.143856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 403/815] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:44.144942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.201162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:44.241435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c4501-8af5-42b9-b08b-a9f832bb5444 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.255361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e707e9-70e2-4294-ad52-f4ddae12379e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.266861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:44.279290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.280558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.280658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.281014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.281938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:44.282334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.283902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.292838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.293749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.294763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.295520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.296628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:44.297167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.298743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.304998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 404/816] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:25:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:44.306079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.308918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 413/817] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 18 10:25:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:44.310285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.418932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35026a-97fd-4078-8247-91bc903d5612 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.423615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0450469-b3c8-426e-a6b2-9f77d378fcb4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.458558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.459238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.459518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.459790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.460530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:44.460862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.462320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.477597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 405/818] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:44.478561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.520240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:44.573002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:44.590214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.591263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.592011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.592638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.593056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b533e65-882a-4a38-af8a-836aaa91a8e0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.593886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.595745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.631090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.631882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.632221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.632500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.644467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.646004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:44.664390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:44.665589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:44.666492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:44.666492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:44.666492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:44.666492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:44.666492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 406/819] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:44.678021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.721358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 414/820] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:25:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:44.722955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.785555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:44.804947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea57b79-351b-4381-b06b-e5886c6a9598 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:44.858684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:44.860438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:44.861090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:44.861762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:44.874368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:44.876590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:44.897469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:44.898498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:25:44.899487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:44.902672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 407/821] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:25:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:44.908972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:44.930105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9645d56-a281-40ca-9aa7-f3f008ef0323 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.000866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cbcc8-b64e-4497-b2d4-d4f0cc0eb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.001570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cbcc8-b64e-4497-b2d4-d4f0cc0eb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.001832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cbcc8-b64e-4497-b2d4-d4f0cc0eb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.002111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cbcc8-b64e-4497-b2d4-d4f0cc0eb2b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.002851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cbcc8-b64e-4497-b2d4-d4f0cc0eb2b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:45.003165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cbcc8-b64e-4497-b2d4-d4f0cc0eb2b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.004563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cbcc8-b64e-4497-b2d4-d4f0cc0eb2b7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.017977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 415/822] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:45.020031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101001-bfc1-4d9c-9189-fecd2d7881bd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.020372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101001-bfc1-4d9c-9189-fecd2d7881bd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.020609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101001-bfc1-4d9c-9189-fecd2d7881bd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.026049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 416/823] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:45.026929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d101001-bfc1-4d9c-9189-fecd2d7881bd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.051904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eea86-4825-4d18-b630-eef2a38fc956 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.086004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:45.096435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c4c9a-693e-4282-b7ac-ca88e50c2342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.097167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c4c9a-693e-4282-b7ac-ca88e50c2342 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.106493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c4c9a-693e-4282-b7ac-ca88e50c2342 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.106493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c4c9a-693e-4282-b7ac-ca88e50c2342 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.106493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c4c9a-693e-4282-b7ac-ca88e50c2342 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:45.106493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c4c9a-693e-4282-b7ac-ca88e50c2342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.106493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c4c9a-693e-4282-b7ac-ca88e50c2342 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.116560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:45.121294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 408/824] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:25:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:45.123469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43640ddb-b84a-43ed-abcf-b6432a65d9f0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.123761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43640ddb-b84a-43ed-abcf-b6432a65d9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.124040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43640ddb-b84a-43ed-abcf-b6432a65d9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.140643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-43640ddb-b84a-43ed-abcf-b6432a65d9f0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:45.142000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43640ddb-b84a-43ed-abcf-b6432a65d9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:45.151146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d101001-bfc1-4d9c-9189-fecd2d7881bd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.192483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.193464 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.194014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.194014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.200975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.202639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.220801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 409/825] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:45.220801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43640ddb-b84a-43ed-abcf-b6432a65d9f0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:45.228174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:45.229206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:45.230108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 417/826] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:45.236772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.330892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43640ddb-b84a-43ed-abcf-b6432a65d9f0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.339447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:45.345857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43676b-98c4-4f08-a6de-6c836dd922a2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.363064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:45.368960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.369612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.369906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.370199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.381932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.382627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.382915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.383195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.383992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:45.384307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.385798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.399283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 418/827] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:45.400198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.454350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.456071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.527644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55220516-2438-484c-9049-4c7ee1f9042a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.567517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.568615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.569080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.569518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.570330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.571864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:45.585407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:45.586558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:45.593637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 419/828] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:45.594502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.618453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:45.694628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 410/829] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:25:45.695304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.712221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54179f-0564-4641-9e93-e4c865d7799a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.741598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:45.751837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.753475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.754007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.754347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/41cc665cbec74e06bfa7acf97f968e52/roles/30ebd4c6517a4a9ca23f5eaf73c732a0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.770557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=41cc665cbec74e06bfa7acf97f968e52, role_id=30ebd4c6517a4a9ca23f5eaf73c732a0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.772109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.820137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e784b-613b-448d-840c-20a532b1f6c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:45.850607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 420/830] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:25:45 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/41cc665cbec74e06bfa7acf97f968e52/roles/30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:45.851447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.868014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:45.869484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:45.869947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:45.870435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:45.871613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:45.872122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:45.874320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:45.882745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 411/831] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:45.890295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:45.960756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:45.998375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27674c3d-9308-4026-aaf9-6e1bc2d8cdf1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.001727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:46.011933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bdda1-bde2-4981-9282-7814e61a0c39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.040240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.041167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.041417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.041690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.054812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.056259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.066759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b90ee-5d3a-4e4b-9a5d-882b7c3d86af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.067730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b90ee-5d3a-4e4b-9a5d-882b7c3d86af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.068099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b90ee-5d3a-4e4b-9a5d-882b7c3d86af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.068382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b90ee-5d3a-4e4b-9a5d-882b7c3d86af None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.069139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b90ee-5d3a-4e4b-9a5d-882b7c3d86af None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:46.069452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b90ee-5d3a-4e4b-9a5d-882b7c3d86af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.070843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b90ee-5d3a-4e4b-9a5d-882b7c3d86af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:46.075169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:46.076707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:46.077832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:46.077832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:46.077832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:46.077832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:46.078066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 412/832] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:46.078137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 421/833] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:46.082111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83b82a-58b9-4234-a6a6-1d5f89bc4282 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.082657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83b82a-58b9-4234-a6a6-1d5f89bc4282 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.082982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.085441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83b82a-58b9-4234-a6a6-1d5f89bc4282 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.088198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 413/834] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:46.094280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75340861-b7cc-4260-9cfc-b9f986f64209 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.094958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75340861-b7cc-4260-9cfc-b9f986f64209 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.095446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75340861-b7cc-4260-9cfc-b9f986f64209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.118860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-75340861-b7cc-4260-9cfc-b9f986f64209 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:46.121195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-75340861-b7cc-4260-9cfc-b9f986f64209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:46.212975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496d35-eb2f-471a-99a4-7c7b852d5a4b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.223067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 414/835] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:25:46.249081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75340861-b7cc-4260-9cfc-b9f986f64209 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.249270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:46.264197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.264908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.265243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.265526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.266268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:46.266583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.268089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.283347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 422/836] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:46.287271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.367646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:46.384694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75340861-b7cc-4260-9cfc-b9f986f64209 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.423720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7369-f022-4479-84fa-013b515916c3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.431916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.433366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.433750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.434123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.447911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:46.459097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:46.460095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:46.461001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 415/837] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:46.466083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.467311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.467631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.468061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.468400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.478238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.480234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:46.502344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:46.504126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:46.505466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:46.509239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 423/838] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:46.510397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.571536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:46.601741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039735-b4c6-49bc-a4ac-2725b240e096 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.643335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e35d2-6e26-4107-bd64-45570fb19486 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.646352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.647293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.647772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.648268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.649305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:46.649858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.651448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.662052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 416/839] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:46.667619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.684055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:46.691401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.692643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.693160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.693655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.695180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:46.695729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.697844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.719889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 424/840] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:46.720426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.791391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdcdf0-2ad3-40d8-bfb8-8d817e795770 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.834318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d8fc1-c36a-448f-8816-c673a4285e3b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:46.835938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:46.838707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.839778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.840258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.840744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.854779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.856852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.871126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:46.872957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:46.873814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:46.874286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:46.875573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=ec899ae414744dde9ec9b76aac1ce9d8) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:46.876007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:46.877806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:46.878210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:46.879139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:46.880029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:46.880029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:46.880029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:46.880029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:46.882544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 417/841] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:46.887841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:46.894640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 425/842] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/role_assignments?user.id=ec899ae414744dde9ec9b76aac1ce9d8&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:46.962489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:47.005790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087c96f-43e1-4ee8-83b3-18154a474133 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:47.046028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:47.046640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:47.046921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:47.047194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:47.047949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:47.048260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:47.049752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:47.063656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 418/843] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:47.069445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:47.181495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566ec72-f5eb-4680-be7e-01f2837ea0e7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:47.218122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:47.218923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:47.219201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:47.219470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:47.227374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:47.228837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:47.243956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:47.244957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:47.245934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 426/844] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:25:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:47.252827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:47.365624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393337d-b53f-46e9-93bb-a77f0f630ca1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:47.391145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:47.403254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:47.404337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:47.404865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:47.405320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:47.406360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:47.406817 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:47.408481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:47.428438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 419/845] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 18 10:25:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:47.434505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:47.467859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:47.547363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ffcdd-3cf2-46b3-8616-9b01e7a125ab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:47.583037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:47.584152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:47.584708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:47.585325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:47.662562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:47.663986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:47.693425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:47.772099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 427/846] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:25:47.777783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:47.888424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad731ced-c45a-4a0a-b58b-28464f5dbda0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:47.925265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880005b-6356-4a91-9d86-1a608b6639ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:47.925983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880005b-6356-4a91-9d86-1a608b6639ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:47.926268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880005b-6356-4a91-9d86-1a608b6639ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:47.926544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880005b-6356-4a91-9d86-1a608b6639ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:47.937796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880005b-6356-4a91-9d86-1a608b6639ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:47.939282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880005b-6356-4a91-9d86-1a608b6639ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4880005b-6356-4a91-9d86-1a608b6639ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:47.959539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:47.960508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:47.961442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:47.961442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:47.961442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:47.961442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:47.964666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 420/847] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:47.967747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:47.994776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90e00c-371c-431b-8274-801e50a979ca None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:47.995099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90e00c-371c-431b-8274-801e50a979ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:47.995350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90e00c-371c-431b-8274-801e50a979ca None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:47.999400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 421/848] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:48.003583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff92e96-fb4e-4b98-990c-40179df332b2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.003876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff92e96-fb4e-4b98-990c-40179df332b2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.004156 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff92e96-fb4e-4b98-990c-40179df332b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.022047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bff92e96-fb4e-4b98-990c-40179df332b2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:48.023578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bff92e96-fb4e-4b98-990c-40179df332b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:48.053904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:48.094597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fc0c0-5412-4f41-b3e6-cc116110b517 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.104903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 422/849] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:25:48.129273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff92e96-fb4e-4b98-990c-40179df332b2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.139601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:48.145821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.146509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.146822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.147102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.151239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:48.151239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:48.151239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:48.171615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 428/850] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:48.173087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.272764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff92e96-fb4e-4b98-990c-40179df332b2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.291937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb0fdf-e329-459c-9e0c-97acb7b56d4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.310948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5236e3-669d-4712-8ca4-49e8e6f92708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.312114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5236e3-669d-4712-8ca4-49e8e6f92708 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.312581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5236e3-669d-4712-8ca4-49e8e6f92708 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.312884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5236e3-669d-4712-8ca4-49e8e6f92708 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.326734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5236e3-669d-4712-8ca4-49e8e6f92708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:48.333051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3cfbb-8591-4ebd-8c48-98333cb339e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.333901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3cfbb-8591-4ebd-8c48-98333cb339e6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.334194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3cfbb-8591-4ebd-8c48-98333cb339e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.334470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3cfbb-8591-4ebd-8c48-98333cb339e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5e5236e3-669d-4712-8ca4-49e8e6f92708 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:48.336691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:48.337981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:48.338853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 423/851] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:48.340359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4505-0c9f-4abd-828f-ea1e4c0379d3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.340359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4505-0c9f-4abd-828f-ea1e4c0379d3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.340534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4505-0c9f-4abd-828f-ea1e4c0379d3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.346271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 424/852] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:48.346271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4505-0c9f-4abd-828f-ea1e4c0379d3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.348420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3cfbb-8591-4ebd-8c48-98333cb339e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:48.351206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3cfbb-8591-4ebd-8c48-98333cb339e6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2ee3cfbb-8591-4ebd-8c48-98333cb339e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:48.368318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:48.369395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 429/853] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0ed8-206a-4cfd-b2ce-8e336befa9ec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.379570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0ed8-206a-4cfd-b2ce-8e336befa9ec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.380211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0ed8-206a-4cfd-b2ce-8e336befa9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.387436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:48.398706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-87ad0ed8-206a-4cfd-b2ce-8e336befa9ec None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:48.399908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87ad0ed8-206a-4cfd-b2ce-8e336befa9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:48.450516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:48.473954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4505-0c9f-4abd-828f-ea1e4c0379d3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.492814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 430/854] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:48.492814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad0ed8-206a-4cfd-b2ce-8e336befa9ec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.521330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.522055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.522354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.522630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.523894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:48.524173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:48.525709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:48.535229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 425/855] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:48.539934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.622033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad0ed8-206a-4cfd-b2ce-8e336befa9ec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.629718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:48.645898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb0ee1-cb4a-484c-acc6-6b2149b71135 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.659509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.660251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.660541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.660846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.661651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:48.661962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:48.663410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:48.677383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 431/856] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:48.678254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.689331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.690021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.690305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.690575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.691771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:48.693636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:48.715483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:48.783476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 426/857] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:25:48 2022] POST /identity/v3/services => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:48.783476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.811634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b41684-75fb-4105-9164-db231b7c2504 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.870192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.871014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.871299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.871574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.891307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:48.894046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:48.914409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48b598-c048-45af-917b-8f575e6479e2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:48.921075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:48.922266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:48.923385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:48.923385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:48.923385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:48.923385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:48.925103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 432/858] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:48.931370 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:48.963964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:48.964753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:48.965051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:48.965466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:48.966696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=642a42337be14f10b410903df57f7527) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:48.967340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:48.968985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:48.980804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 427/859] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:48 2022] GET /identity/v3/role_assignments?user.id=642a42337be14f10b410903df57f7527&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:49.043248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:49.061564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f814-7969-4474-9f80-bebe4563a34c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:49.103623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d68ad4-5d29-4ad7-8ac3-7a2d5208c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:49.104315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d68ad4-5d29-4ad7-8ac3-7a2d5208c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.104605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d68ad4-5d29-4ad7-8ac3-7a2d5208c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.104907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d68ad4-5d29-4ad7-8ac3-7a2d5208c3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.105655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d68ad4-5d29-4ad7-8ac3-7a2d5208c3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:49.105963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d68ad4-5d29-4ad7-8ac3-7a2d5208c3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:49.107423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d68ad4-5d29-4ad7-8ac3-7a2d5208c3d1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:49.121433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 433/860] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:49.127843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:49.144814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:49.186063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635de4c-77f5-4ed2-b978-cb7c4309194f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.186403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635de4c-77f5-4ed2-b978-cb7c4309194f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.186628 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635de4c-77f5-4ed2-b978-cb7c4309194f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.188374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 434/861] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:49.194368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe81a3-f890-4c78-82d9-b11392fc6c1d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.194657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe81a3-f890-4c78-82d9-b11392fc6c1d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.194934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe81a3-f890-4c78-82d9-b11392fc6c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.213631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9bfe81a3-f890-4c78-82d9-b11392fc6c1d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:49.215096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9bfe81a3-f890-4c78-82d9-b11392fc6c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:49.253675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af26c5-de98-4d2b-8a48-5225bede7433 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:49.290743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:49.291563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.291888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.292235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.299512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 435/862] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:49.308258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:49.311027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:49.327850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:49.328875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:49.329833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:49.332456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 428/863] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:25:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:49.334321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe81a3-f890-4c78-82d9-b11392fc6c1d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:49.340302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:49.428182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:49.433825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:49.455296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe81a3-f890-4c78-82d9-b11392fc6c1d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:49.483779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393555-5437-4bfc-96c6-3938cfc26b30 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:49.494168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:49.495005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.495289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.495563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.496226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:49.498199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:49.511241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:49.512214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:49.513581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:49.516546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 436/864] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Jul 18 10:25:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:49.519794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:49.533082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:49.534562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.535332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.535811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.537227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:49.537549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:49.539025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:49.565350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 429/865] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 18 10:25:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:49.576822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:49.650022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755863d1-230b-4011-8515-9b8c0681e57b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:49.690957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192d00-f69e-4ad7-acbc-42db64cd8fff None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:49.692796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:49.694502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.694502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.694502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.695072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:49.695387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:49.696836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:49.709129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 437/866] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:49.710579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:49.716367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:49.724711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:49.734981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:49.735638 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.735974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.736270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.816096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:49.817741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:49.858860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641917f-dffd-4203-a26f-e5a61e588890 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:49.910420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:49.911635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:49.912074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:49.912931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:49.913804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:49.914280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:49.916393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:49.929145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 438/867] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:49.939725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 430/868] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:49 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:25:49.941220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:50.018710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:50.062242 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a417e-7e2d-48ea-942e-d940c729385c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:50.099557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:50.100279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:50.100561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:50.100859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:50.114345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:50.115913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:50.136437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:50.137426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:50.138401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:50.138401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:50.138401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:50.138401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:50.139140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 439/869] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:50.145863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:50.178219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:50.261870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffded6-dace-487f-91af-74e46b4ed4c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:50.301299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:50.302788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:50.303277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:50.303776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:50.304715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:50.305290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:50.306894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:50.320316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 431/870] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:50.329868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:50.445456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59d19-e0ce-47a7-9f5c-46292a8b9c24 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:50.464298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:50.485768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:50.486579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:50.486857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:50.487130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:50.494577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:50.496065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:50.512723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:50.513895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:50.514819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 440/871] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:50.520274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:50.627234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:50.642416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f0ef2-6232-495c-b233-2564c11363da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:50.680850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d7e71-2c18-4be6-b1b2-2d16e520b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:50.681527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d7e71-2c18-4be6-b1b2-2d16e520b165 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:50.681815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d7e71-2c18-4be6-b1b2-2d16e520b165 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:50.682158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d7e71-2c18-4be6-b1b2-2d16e520b165 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:50.682967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d7e71-2c18-4be6-b1b2-2d16e520b165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:50.683282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d7e71-2c18-4be6-b1b2-2d16e520b165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:50.685694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d7e71-2c18-4be6-b1b2-2d16e520b165 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:50.699767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 432/872] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:50.706849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:50.820107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3f538-ecb5-4c04-9e87-28a4dab0ae99 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:50.858612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c233a1e-31ca-497f-9afc-37e15d6c10c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:50.859686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233a1e-31ca-497f-9afc-37e15d6c10c2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:50.860200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233a1e-31ca-497f-9afc-37e15d6c10c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:50.860693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233a1e-31ca-497f-9afc-37e15d6c10c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:50.861749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c233a1e-31ca-497f-9afc-37e15d6c10c2 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=41cc665cbec74e06bfa7acf97f968e52) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:50.862398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c233a1e-31ca-497f-9afc-37e15d6c10c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:50.864050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c233a1e-31ca-497f-9afc-37e15d6c10c2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:50.870969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 441/873] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:50 2022] GET /identity/v3/role_assignments?user.id=41cc665cbec74e06bfa7acf97f968e52&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:51.060767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196357f-db31-4c2e-8978-74db712b0a11 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.061438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196357f-db31-4c2e-8978-74db712b0a11 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.061950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196357f-db31-4c2e-8978-74db712b0a11 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.063886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 433/874] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:51.069891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc2010-8cc0-4d45-b27f-ac441b9dd8fb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.070803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc2010-8cc0-4d45-b27f-ac441b9dd8fb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.071301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc2010-8cc0-4d45-b27f-ac441b9dd8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.089434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-defc2010-8cc0-4d45-b27f-ac441b9dd8fb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:51.090737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-defc2010-8cc0-4d45-b27f-ac441b9dd8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:51.132996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:51.167668 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 442/875] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:51.192556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196357f-db31-4c2e-8978-74db712b0a11 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.221753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9929a7-c1fc-499c-a10c-12dbe789bcfa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.221977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9929a7-c1fc-499c-a10c-12dbe789bcfa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.222235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9929a7-c1fc-499c-a10c-12dbe789bcfa None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.223892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 443/876] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:51.229276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5c1c-a2d7-4ad1-b51e-8cb4b4b2fe1b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.229419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5c1c-a2d7-4ad1-b51e-8cb4b4b2fe1b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.229660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5c1c-a2d7-4ad1-b51e-8cb4b4b2fe1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.247323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-daab5c1c-a2d7-4ad1-b51e-8cb4b4b2fe1b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:51.248539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-daab5c1c-a2d7-4ad1-b51e-8cb4b4b2fe1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:51.318286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196357f-db31-4c2e-8978-74db712b0a11 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:51.342110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 444/877] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:51.359887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:51.361049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.361727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.362219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.363080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:51.364442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab5c1c-a2d7-4ad1-b51e-8cb4b4b2fe1b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.366182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:51.379103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:51.380152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:51.381078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:51.384012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 434/878] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:51.390023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.404438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:51.494834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab5c1c-a2d7-4ad1-b51e-8cb4b4b2fe1b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:51.497762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:51.506409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bd84e-6fc5-4b56-92f2-5afef9093267 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:51.541431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:51.542659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.543151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.543636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.544763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:51.546502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:51.546738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:51.547866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.548223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.548577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.549392 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:51.549885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:51.551467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:51.561482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 435/879] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:51.562480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:51.562707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:51.563660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:25:51.564513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:51.571134 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 445/880] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:51.573211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.673042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:51.693820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee24570-f43d-42b5-9d38-fdac9412ae6c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:51.704971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53394d70-6690-4daa-a7e7-fb394c1c1425 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:51.742622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:51.743813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.744341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.744892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.746037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:51.746533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:51.746837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:51.748203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.748563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.748987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.749720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:51.749843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:51.753254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:51.756233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:51.764973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 446/881] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:51.765579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 436/882] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:51.767566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.768807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.812887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:51.905071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38fa89-4086-4aa2-9e3a-2489819fafde None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:51.906962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fd9b3-410b-4c0a-9338-f388df4f8eae None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:51.947465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56238ac8-4885-4655-a8a5-9985e57f9670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:51.948538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:51.948898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56238ac8-4885-4655-a8a5-9985e57f9670 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.949230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:51.949520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.949802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.950663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56238ac8-4885-4655-a8a5-9985e57f9670 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:51.951159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56238ac8-4885-4655-a8a5-9985e57f9670 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:51.951473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] RBAC: Adding query filter params (interface=public, service_id=556cac1b550248cb8a51de759ca65ee9, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:51.951908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:51.952588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56238ac8-4885-4655-a8a5-9985e57f9670 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:51.953126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56238ac8-4885-4655-a8a5-9985e57f9670 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:51.953349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:51.954927 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56238ac8-4885-4655-a8a5-9985e57f9670 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:51.960819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 437/883] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/endpoints?service_id=556cac1b550248cb8a51de759ca65ee9&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:51.965147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 447/884] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:51.966641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:51.973536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:52.089153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae3129-9e4c-4d91-94e6-9d1982953648 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:52.117107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fc264-dd04-4b2e-94fe-528af54f95b6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:52.117669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fc264-dd04-4b2e-94fe-528af54f95b6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:52.117669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fc264-dd04-4b2e-94fe-528af54f95b6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:52.119347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 448/885] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:52.125836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dfce2-e544-4799-a9bd-ca842fc0206e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:52.126167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dfce2-e544-4799-a9bd-ca842fc0206e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:52.126401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dfce2-e544-4799-a9bd-ca842fc0206e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:52.138012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:52.139891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:52.140473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:52.140998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:52.141975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:52.143992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:52.146871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8d5dfce2-e544-4799-a9bd-ca842fc0206e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:52.148088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d5dfce2-e544-4799-a9bd-ca842fc0206e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:52.152506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 438/886] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:25:51 2022] GET /identity/v3/services => generated 1011 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:25:52.235693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 449/887] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:25:52.258983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:52.337685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:52.375971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3be59-64c7-45d5-8a66-cedca115e6d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:52.412052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7739ddc-2c38-4b2d-9db1-4ff2f76b26be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:52.412720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7739ddc-2c38-4b2d-9db1-4ff2f76b26be None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:52.413023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7739ddc-2c38-4b2d-9db1-4ff2f76b26be None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:52.413296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7739ddc-2c38-4b2d-9db1-4ff2f76b26be None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:52.414089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7739ddc-2c38-4b2d-9db1-4ff2f76b26be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:52.415616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7739ddc-2c38-4b2d-9db1-4ff2f76b26be None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:52.509168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 439/888] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:25:52 2022] POST /identity/v3/services => generated 269 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:53.078719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829a35-bcc8-4d47-bf93-31976ee69f0c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:53.079063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829a35-bcc8-4d47-bf93-31976ee69f0c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:53.079185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829a35-bcc8-4d47-bf93-31976ee69f0c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:53.080830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 450/889] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:53.086195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803208be-10fb-4442-bd8d-d5fcba05dd3b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:53.086486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803208be-10fb-4442-bd8d-d5fcba05dd3b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:53.086761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803208be-10fb-4442-bd8d-d5fcba05dd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:53.104276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-803208be-10fb-4442-bd8d-d5fcba05dd3b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:53.105503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-803208be-10fb-4442-bd8d-d5fcba05dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:53.188904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 440/890] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:53.212449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829a35-bcc8-4d47-bf93-31976ee69f0c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:53.246338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:53.329782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829a35-bcc8-4d47-bf93-31976ee69f0c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:53.365491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:53.366308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:53.366674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:53.366956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:53.378329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:53.386630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:53.387612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:53.388462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 451/891] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 18 10:25:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:53.394003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803208be-10fb-4442-bd8d-d5fcba05dd3b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:53.450588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:53.504917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803208be-10fb-4442-bd8d-d5fcba05dd3b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:53.539785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:53.540457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:53.540740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:53.541046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:53.541798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:53.542106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:53.543552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:53.559087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:53.560012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 441/892] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 18 10:25:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:53.650301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:53.667222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6018e-70cd-4da9-b2d0-845967bfbf94 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:53.706435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:53.707231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:53.707512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:53.707784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:53.722408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:53.723900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:53.743252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:53.744216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:53.745180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:53.745180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:53.745180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:53.745180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:53.745180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 452/893] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:53.751229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:53.871236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb240c-096c-49de-b691-60db2ee502b5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:53.887711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:53.909686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:53.910298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:53.910576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:53.910848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:53.911573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:53.911879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:53.913291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:53.926759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 442/894] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:53.932562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:54.041786 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e5ea0-3ee6-46fb-8754-dc525a680b43 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:54.078889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd405b-c164-4140-9e8d-da6354b61dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:54.079704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd405b-c164-4140-9e8d-da6354b61dcb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.079984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd405b-c164-4140-9e8d-da6354b61dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.080296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd405b-c164-4140-9e8d-da6354b61dcb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.083196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:54.088540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd405b-c164-4140-9e8d-da6354b61dcb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:54.090207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd405b-c164-4140-9e8d-da6354b61dcb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b0bd405b-c164-4140-9e8d-da6354b61dcb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:54.105772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:54.107598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:54.108546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 453/895] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:25:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:54.114079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:54.217899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070481-7dca-4912-a206-7e8729280d0f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.218419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070481-7dca-4912-a206-7e8729280d0f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.218419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070481-7dca-4912-a206-7e8729280d0f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.220850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 454/896] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:54.227936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4da7-5784-4a81-a7f7-81bfb5d43eb1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.228331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4da7-5784-4a81-a7f7-81bfb5d43eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.228757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4da7-5784-4a81-a7f7-81bfb5d43eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.229107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda8ff0-27fb-4e6f-a96f-adc037b1993b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:54.251964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1cbd4da7-5784-4a81-a7f7-81bfb5d43eb1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:54.253697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1cbd4da7-5784-4a81-a7f7-81bfb5d43eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:54.273049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531d23-143a-4e2e-b38e-a7f82811afd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:54.273724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531d23-143a-4e2e-b38e-a7f82811afd8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.274007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531d23-143a-4e2e-b38e-a7f82811afd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.274280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531d23-143a-4e2e-b38e-a7f82811afd8 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.275119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531d23-143a-4e2e-b38e-a7f82811afd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:54.275424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531d23-143a-4e2e-b38e-a7f82811afd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:54.276934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531d23-143a-4e2e-b38e-a7f82811afd8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:54.300918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 443/897] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 18 10:25:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:54.300918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616a574-4539-4846-aced-bd0c3127edd8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.300918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616a574-4539-4846-aced-bd0c3127edd8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.300918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616a574-4539-4846-aced-bd0c3127edd8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.300918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 444/898] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:54.303061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616a574-4539-4846-aced-bd0c3127edd8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:54.335701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 455/899] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:54.339985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:54.343779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cfd15-e17f-4f3a-9fe6-9c4badee9c58 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.344289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cfd15-e17f-4f3a-9fe6-9c4badee9c58 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.344824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cfd15-e17f-4f3a-9fe6-9c4badee9c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.364603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3a7cfd15-e17f-4f3a-9fe6-9c4badee9c58 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:54.365840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3a7cfd15-e17f-4f3a-9fe6-9c4badee9c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:54.423771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616a574-4539-4846-aced-bd0c3127edd8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:54.439644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 456/900] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:54.440362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cfd15-e17f-4f3a-9fe6-9c4badee9c58 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:54.478178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:54.478746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.479034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.479310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.504222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:54.550372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cfd15-e17f-4f3a-9fe6-9c4badee9c58 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:54.555775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:54.557355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:54.589165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788adfe-5315-4c81-99d1-b9b543c43661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:54.590186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788adfe-5315-4c81-99d1-b9b543c43661 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.591069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788adfe-5315-4c81-99d1-b9b543c43661 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.591345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788adfe-5315-4c81-99d1-b9b543c43661 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.592507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788adfe-5315-4c81-99d1-b9b543c43661 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:54.593568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788adfe-5315-4c81-99d1-b9b543c43661 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:54.594306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:54.666598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 445/901] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:25:54 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:25:54.667545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:54.676853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 457/902] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:25:54 2022] POST /identity/v3/services => generated 241 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:54.678325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240e9e4-af35-4252-b523-d5de0e39813d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.678682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240e9e4-af35-4252-b523-d5de0e39813d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.679027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240e9e4-af35-4252-b523-d5de0e39813d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.680708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 458/903] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:54.681420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240e9e4-af35-4252-b523-d5de0e39813d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:54.789600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9cc58-d522-44d9-aeb9-f47326a2e17b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:54.821508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240e9e4-af35-4252-b523-d5de0e39813d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:54.825085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:54.830786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df3cb1-dda5-41f6-8096-a5a3670412aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:54.831938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3cb1-dda5-41f6-8096-a5a3670412aa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.832442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3cb1-dda5-41f6-8096-a5a3670412aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.832675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3cb1-dda5-41f6-8096-a5a3670412aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.833410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df3cb1-dda5-41f6-8096-a5a3670412aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:54.834818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df3cb1-dda5-41f6-8096-a5a3670412aa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:54.839786 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d9df3cb1-dda5-41f6-8096-a5a3670412aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:54.849790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:54.850822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:54.853047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 446/904] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:54.853859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276f00-4ab8-44e9-b9d6-e2464703e9e1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.853859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276f00-4ab8-44e9-b9d6-e2464703e9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.853859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276f00-4ab8-44e9-b9d6-e2464703e9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.865602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:54.866411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:54.866688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:54.866963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:54.875557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-66276f00-4ab8-44e9-b9d6-e2464703e9e1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:54.878709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66276f00-4ab8-44e9-b9d6-e2464703e9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:54.881331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:54.882848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:54.902252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:54.904049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:54.905491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:54.905491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:54.905491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:54.905491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:54.905491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 459/905] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:54.905491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:54.957597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 447/906] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:54.958931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276f00-4ab8-44e9-b9d6-e2464703e9e1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.022014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb6635-d015-403e-9b19-2e19b17854d8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.060206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.060801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.061100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.061376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.062097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:55.062403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.063779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.074392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 460/907] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:55.074392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.082786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276f00-4ab8-44e9-b9d6-e2464703e9e1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.103671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:55.113884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:55.123887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.124565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.124878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.126736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.126736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:55.126736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.127733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.139643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 448/908] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:55.140316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.191123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa1d25-5726-482b-91d8-fadc3d71860c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.228682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.230288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.230768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.231951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.232824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.234471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:55.248675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:55.249708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:55.250557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:55.253061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 461/909] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:55.253982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.254099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177dddb-725a-4b9d-8b07-3f5c164138bf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.292937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.294239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.294527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.294798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.295672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:55.295998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.297403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.303558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 449/910] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:55.304631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.356706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:55.362150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:55.366045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4de32-f029-4a40-b85b-420ad74c4df8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.403043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.404243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.404722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.405173 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.413041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.414827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.417067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d821f-ed6a-4708-b19a-572fa45c9fd4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:55.430830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:55.431936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:55.433097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:55.435890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 462/911] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:55.436834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.460203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.460890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.461178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.461453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.462165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:55.462470 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.463912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.469386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 450/912] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:55.470182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.552140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f578713-57c7-4c0a-8847-bdeb3d9a0a15 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.584385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add13ee-99dd-4170-ac9d-e1e5b5190aa5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.597186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.598293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.599033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:55.599319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.599614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.600376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.602244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.620770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.621411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.622363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.622867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.623748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:55.624057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.625526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.639423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 451/913] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:55.640270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.693770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 463/914] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:25:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:55.694643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.716832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:55.784570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bb283-23f7-44ff-a1ec-4c3395aa8448 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.850104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4b1ea-b021-4028-b24e-50b7fe60cf42 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:55.855813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.857022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.857422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.857800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.858655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:55.859071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.861423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.867395 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 452/915] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:55.874604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:55.909804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:55.909804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:55.910265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:55.910460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:55.911311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] RBAC: Adding query filter params (role.id=5c9ff5726a4f4a3c86b6504b08ec2239, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=41cc665cbec74e06bfa7acf97f968e52) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:55.911706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:55.913100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:55.919019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 464/916] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/role_assignments?user.id=41cc665cbec74e06bfa7acf97f968e52&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=5c9ff5726a4f4a3c86b6504b08ec2239 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:55.931469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:56.008924 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7d859-9b68-4c54-8f01-b4dca1fcd94a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:56.048476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b392e-5b83-4d49-add6-95603ffbc3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:56.049187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b392e-5b83-4d49-add6-95603ffbc3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:56.049472 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b392e-5b83-4d49-add6-95603ffbc3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:56.049790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b392e-5b83-4d49-add6-95603ffbc3b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:56.050604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b392e-5b83-4d49-add6-95603ffbc3b8 None admin] RBAC: Adding query filter params (interface=public, service_id=cd879db62ee045b0b6231f7877f92851, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:56.050920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b392e-5b83-4d49-add6-95603ffbc3b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:56.052319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b392e-5b83-4d49-add6-95603ffbc3b8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:56.058737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 453/917] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:25:55 2022] GET /identity/v3/endpoints?service_id=cd879db62ee045b0b6231f7877f92851&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:56.065946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:56.145726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:56.183282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57308-e409-4a60-a4ae-74c49228818a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:56.219918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abb71f-891f-4e57-aaa2-8573a96f19cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:56.220865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abb71f-891f-4e57-aaa2-8573a96f19cf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:56.221299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abb71f-891f-4e57-aaa2-8573a96f19cf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:56.221720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abb71f-891f-4e57-aaa2-8573a96f19cf None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:56.222583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abb71f-891f-4e57-aaa2-8573a96f19cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:56.224104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abb71f-891f-4e57-aaa2-8573a96f19cf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:56.231509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 465/918] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:25:56 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:25:56.792323 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e295-40c2-467c-8cd1-e020d3757518 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:56.792865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e295-40c2-467c-8cd1-e020d3757518 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:56.792979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e295-40c2-467c-8cd1-e020d3757518 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:56.795003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 454/919] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:56.802572 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a231a48-14c3-44d5-83ff-2429c7aaaa30 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:56.803312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a231a48-14c3-44d5-83ff-2429c7aaaa30 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:56.803312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a231a48-14c3-44d5-83ff-2429c7aaaa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:56.823250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7a231a48-14c3-44d5-83ff-2429c7aaaa30 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:56.824845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7a231a48-14c3-44d5-83ff-2429c7aaaa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:56.895496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 466/920] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:56.917283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8e295-40c2-467c-8cd1-e020d3757518 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:57.025902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:57.030582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8e295-40c2-467c-8cd1-e020d3757518 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:57.067801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:57.068523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:57.068825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:57.069103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:57.069760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:57.071148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:57.083634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:57.084662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:57.085535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 455/921] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:57.091337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a231a48-14c3-44d5-83ff-2429c7aaaa30 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:57.159355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:57.200403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a231a48-14c3-44d5-83ff-2429c7aaaa30 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:57.236709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:57.237446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:57.237688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:57.237965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:57.238677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:57.238984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:57.240364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:57.245751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 467/922] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:25:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:57.251769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:57.360001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672766-dcb5-4bdb-8f96-cf10d99d306b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:57.395542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:57.396236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:57.396512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:57.396801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:57.397542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:57.397846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:57.399252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:57.406135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 456/923] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:25:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:57.411631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:57.514770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d612-0326-4c0c-8938-0e5c350811e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:57.541609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:57.550458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd61109-8049-42f4-b01a-f65e096f4622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:57.551118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd61109-8049-42f4-b01a-f65e096f4622 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:57.551408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd61109-8049-42f4-b01a-f65e096f4622 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:57.551682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd61109-8049-42f4-b01a-f65e096f4622 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:57.552479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd61109-8049-42f4-b01a-f65e096f4622 None admin] RBAC: Adding query filter params (interface=public, service_id=245b2e848318421497f55fce9362103c, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:57.558386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd61109-8049-42f4-b01a-f65e096f4622 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:57.558386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd61109-8049-42f4-b01a-f65e096f4622 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:57.562427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 468/924] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:25:57 2022] GET /identity/v3/endpoints?service_id=245b2e848318421497f55fce9362103c&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:57.567968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:57.597951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:57.675512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6255f5-d36c-47eb-b5f1-7743b865df50 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:57.710963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25292d16-2945-4be3-8a5b-c4ec09e630b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:57.711626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25292d16-2945-4be3-8a5b-c4ec09e630b0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:57.711912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25292d16-2945-4be3-8a5b-c4ec09e630b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:57.712190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25292d16-2945-4be3-8a5b-c4ec09e630b0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:57.712905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25292d16-2945-4be3-8a5b-c4ec09e630b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:57.714295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25292d16-2945-4be3-8a5b-c4ec09e630b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:57.721424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 457/925] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:25:57 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:25:57.949107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28b5d-8e0e-4ae0-83f9-b099a2ce71d8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:57.949525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28b5d-8e0e-4ae0-83f9-b099a2ce71d8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:57.950311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28b5d-8e0e-4ae0-83f9-b099a2ce71d8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:57.951943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 469/926] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:57.957398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acb925-3a6a-4c3f-b357-5e535e8b1073 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:57.957846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acb925-3a6a-4c3f-b357-5e535e8b1073 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:57.958027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acb925-3a6a-4c3f-b357-5e535e8b1073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:57.977486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-30acb925-3a6a-4c3f-b357-5e535e8b1073 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:57.978668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-30acb925-3a6a-4c3f-b357-5e535e8b1073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:58.050649 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 458/927] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:58.075245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28b5d-8e0e-4ae0-83f9-b099a2ce71d8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.109633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729a924-1165-4164-97f3-b25aecfaacb2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.109970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729a924-1165-4164-97f3-b25aecfaacb2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.110288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729a924-1165-4164-97f3-b25aecfaacb2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.111942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 459/928] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:58.117374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c963d75-836b-4f57-8377-de334b8e2623 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.117662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c963d75-836b-4f57-8377-de334b8e2623 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.117937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c963d75-836b-4f57-8377-de334b8e2623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.138811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c963d75-836b-4f57-8377-de334b8e2623 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:58.140906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c963d75-836b-4f57-8377-de334b8e2623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:58.159903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:58.194098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28b5d-8e0e-4ae0-83f9-b099a2ce71d8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.230748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 460/929] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:58.239099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:58.239779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.240136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.240422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.241209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:58.242675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:58.244070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING keystone.server.flask.common [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 10:25:58.255166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c963d75-836b-4f57-8377-de334b8e2623 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.306805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.sql.core [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:25:58.306805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 10:25:58.306805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [parameters: {'id': 'cf667258077d4391b419ff0dc1c5a2c6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 10:25:58.306805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cf667258077d4391b419ff0dc1c5a2c6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cf667258077d4391b419ff0dc1c5a2c6. Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:25:58.317813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 10:25:58.332577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application yield resource Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise self.value Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application session.commit() Jul 18 10:25:58.333622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:25:58.334526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:25:58.335426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:25:58.336253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [parameters: {'id': 'cf667258077d4391b419ff0dc1c5a2c6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:58.338652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:58.340745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cf667258077d4391b419ff0dc1c5a2c6. Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 470/930] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Jul 18 10:25:58 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:58.342976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.380190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:58.381007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c963d75-836b-4f57-8377-de334b8e2623 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.417783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e584914-4670-406a-8620-4923bde6b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:58.419523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e584914-4670-406a-8620-4923bde6b930 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.420081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e584914-4670-406a-8620-4923bde6b930 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.420558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e584914-4670-406a-8620-4923bde6b930 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.432600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e584914-4670-406a-8620-4923bde6b930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:25:58.440650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a2b91-8e6e-45ab-9c52-f35d3d437857 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1e584914-4670-406a-8620-4923bde6b930 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:25:58.441419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:58.442585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:58.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 461/931] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 18 10:25:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:58.444600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffeae9-6a34-4490-8f6b-2247521b1c0e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.444600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffeae9-6a34-4490-8f6b-2247521b1c0e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.444748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffeae9-6a34-4490-8f6b-2247521b1c0e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.446198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 462/932] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:25:58.451865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffeae9-6a34-4490-8f6b-2247521b1c0e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.466365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:58.482706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbaa43-647b-4992-a41a-2648d5b82a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:58.483542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbaa43-647b-4992-a41a-2648d5b82a98 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.483923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbaa43-647b-4992-a41a-2648d5b82a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.484272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbaa43-647b-4992-a41a-2648d5b82a98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.492655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbaa43-647b-4992-a41a-2648d5b82a98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:58.494551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbaa43-647b-4992-a41a-2648d5b82a98 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-eafbaa43-647b-4992-a41a-2648d5b82a98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:58.510255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:58.511399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:58.513129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:58.514151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 471/933] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:58.515465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990c933-81fa-460d-93fb-3b41cb67b53c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.515757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990c933-81fa-460d-93fb-3b41cb67b53c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.516123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990c933-81fa-460d-93fb-3b41cb67b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.535070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c990c933-81fa-460d-93fb-3b41cb67b53c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:58.536621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c990c933-81fa-460d-93fb-3b41cb67b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:58.568151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffeae9-6a34-4490-8f6b-2247521b1c0e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.605205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:58.606505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.606802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.607075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.607898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:58.608204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:58.609651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:58.610200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 472/934] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:25:58.611264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990c933-81fa-460d-93fb-3b41cb67b53c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.625852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 463/935] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 18 10:25:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:58.632838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.664261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:58.710560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:58.737088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990c933-81fa-460d-93fb-3b41cb67b53c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.740103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac5758-b0ce-4ce3-91ae-89783dcb6319 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.776136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:58.776964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.777332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.778103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.778870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:58.779175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:58.780614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:58.781590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:58.782388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:58.782675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:58.782954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:58.794115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:58.795782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:58.809408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 473/936] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:25:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:58.810312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:58.825094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:58.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:58.827066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:58.827066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:58.827066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:58.827066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:58.827066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 464/937] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:25:58.833975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:58.968763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d2bc-36e1-43cd-b836-a2c2dff5f623 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.972770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:58.982077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff01a81-8d32-417b-acd1-637347c384f7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:58.999137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:59.016235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.017363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.017966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.018249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.019088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:59.019405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.020826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.027118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.028201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.028493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.028793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.030375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.031811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.034711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 465/938] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:59.040136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:59.051025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:59.052057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:59.053389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 474/939] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Jul 18 10:25:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:59.058594 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.156243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343ef64-7040-4c90-b277-546e17ab542d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.181041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c25d9-67c1-4ad6-841a-60994b7451a6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.192152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.193095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.193389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.193662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.202685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.204268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.218672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.219344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.219633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.219913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:25:59.220016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:59.220947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:25:59.221899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:59.222444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:59.222444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.222444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.226120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 466/940] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:59.227397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.228017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 475/941] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:59.234873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.263965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:59.281474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:59.342129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e08d50-4fa5-40e6-8c43-0d148f5898f0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.347187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd63c9a-cefd-4560-9a82-4bdfd5721752 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.379719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.380253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.380586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.380983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.381806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:59.382109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.383550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.385793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.387320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.387855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.388330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.389294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:59.389759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.391357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.402004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 476/942] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:59.403281 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.408695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 467/943] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:59.409958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.504079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:59.518320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baadcc-7b5f-4493-b3eb-a1fcf3e90e94 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.523407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3a3f-ad2b-47f3-8111-cabcd9d40699 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.525610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:59.555775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.556448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.558539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.558539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.558539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.559620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.563808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.564862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.565331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.565752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.577096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.581444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:25:59.598263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:59.599261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:59.600263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:25:59.600263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:25:59.600263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:25:59.600263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:25:59.604069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 468/944] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:59.605360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.689849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 477/945] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:25:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:25:59.720203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7641e19-3a59-4eff-87d4-82657511c374 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.768651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f80b3-5c70-4274-8770-9098733acd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.769329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f80b3-5c70-4274-8770-9098733acd8e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.769615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f80b3-5c70-4274-8770-9098733acd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.769937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f80b3-5c70-4274-8770-9098733acd8e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.770672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f80b3-5c70-4274-8770-9098733acd8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:25:59.770979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f80b3-5c70-4274-8770-9098733acd8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.772452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f80b3-5c70-4274-8770-9098733acd8e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.790524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 469/946] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:25:59.791969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:25:59.897282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27ff5b-f508-47f6-b390-00a94199343b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.897497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27ff5b-f508-47f6-b390-00a94199343b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.898463 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27ff5b-f508-47f6-b390-00a94199343b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.901496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 470/947] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:25:59.905904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53ece3-dfa5-4507-8db0-bdc09b02c357 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.906297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53ece3-dfa5-4507-8db0-bdc09b02c357 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.906652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53ece3-dfa5-4507-8db0-bdc09b02c357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.917198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e74432-3b19-474a-b44b-a6da7df9f928 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:25:59.925490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f53ece3-dfa5-4507-8db0-bdc09b02c357 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:25:59.926937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f53ece3-dfa5-4507-8db0-bdc09b02c357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:25:59.931287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:59.961144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:25:59.961958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:25:59.962264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:25:59.962543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:25:59.971761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:25:59.973294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:25:59.989755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:25:59.990694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:25:59.991919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 478/948] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:25:59.998415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:00.004029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:00.019912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 471/949] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:25:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:00.045340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53ece3-dfa5-4507-8db0-bdc09b02c357 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:00.130976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bd73-f570-4844-8696-767585e42294 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:00.179920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53ece3-dfa5-4507-8db0-bdc09b02c357 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:00.185509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:00.186478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:00.187048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:00.187621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:00.189169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:00.189705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:00.192321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:00.210438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 479/950] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:00.216662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:00.220506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:00.221599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:00.222207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:00.222693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:00.223579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:00.225242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:00.239177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:00.240460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:00.241375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:00.244311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 472/951] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:26:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:00.252856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:00.254420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:00.298066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:00.348448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb134-c946-4b0f-8ba6-69e728349824 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:00.365354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e29b18-5a3e-4cc2-8384-ac32f9f0f673 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:00.395518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:00.396425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:00.396716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:00.397184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/41cc665cbec74e06bfa7acf97f968e52/roles/5c9ff5726a4f4a3c86b6504b08ec2239` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:00.400192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=41cc665cbec74e06bfa7acf97f968e52, role_id=5c9ff5726a4f4a3c86b6504b08ec2239)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:00.401846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:00.407996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:00.409092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:00.409659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:00.410142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:00.411146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:00.411661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:00.413501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:00.424803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 473/952] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:26:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:00.425836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:00.500818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 480/953] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:26:00 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/41cc665cbec74e06bfa7acf97f968e52/roles/5c9ff5726a4f4a3c86b6504b08ec2239 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:26:00.515402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:00.555861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a63f3-ff92-4d83-b228-416f88e3335b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:00.595391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e27007-0098-4b75-9dd3-3f12d229415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:00.596129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e27007-0098-4b75-9dd3-3f12d229415b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:00.596415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e27007-0098-4b75-9dd3-3f12d229415b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:00.596744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e27007-0098-4b75-9dd3-3f12d229415b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:00.597507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e27007-0098-4b75-9dd3-3f12d229415b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:00.597874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e27007-0098-4b75-9dd3-3f12d229415b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:00.599310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e27007-0098-4b75-9dd3-3f12d229415b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:00.605940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 474/954] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:26:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:00.613290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:00.729715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8e8ca-da07-4ba9-a157-ddcbef5b9332 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:00.756961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:00.769422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cd92a-e122-4d9a-89f8-e64d20f1bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:00.770510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cd92a-e122-4d9a-89f8-e64d20f1bdc5 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:00.771008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cd92a-e122-4d9a-89f8-e64d20f1bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:00.771487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cd92a-e122-4d9a-89f8-e64d20f1bdc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:00.772514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cd92a-e122-4d9a-89f8-e64d20f1bdc5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:00.774246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cd92a-e122-4d9a-89f8-e64d20f1bdc5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:00.908973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 481/955] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:26:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:01.018954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facfe5a-0699-47b4-b869-dbd130b0b0aa None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:01.019292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facfe5a-0699-47b4-b869-dbd130b0b0aa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:01.019513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facfe5a-0699-47b4-b869-dbd130b0b0aa None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:01.021268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 475/956] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:01.027843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a08754-df22-422b-9c4f-f5d849676ba9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:01.028142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a08754-df22-422b-9c4f-f5d849676ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:01.028422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a08754-df22-422b-9c4f-f5d849676ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:01.048582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a4a08754-df22-422b-9c4f-f5d849676ba9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:01.049841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a4a08754-df22-422b-9c4f-f5d849676ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:01.134355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 482/957] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:01.161724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facfe5a-0699-47b4-b869-dbd130b0b0aa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:01.270782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facfe5a-0699-47b4-b869-dbd130b0b0aa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:01.303481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:01.309518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:01.310200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:01.310486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:01.310764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:01.311492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:01.313145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:01.314660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING keystone.server.flask.common [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 10:26:01.380426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:26:01.380426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 10:26:01.380426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [parameters: {'user_id': 'ded66e5055454adaa018303cc2bb4246', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 10:26:01.380426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jul 18 10:26:01.398072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:26:01.399023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:26:01.400015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:26:01.401138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:26:01.403065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:26:01.404958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ded66e5055454adaa018303cc2bb4246', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:01.406948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 10:26:01.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 10:26:01.411462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 10:26:01.411462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 10:26:01.411462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:26:01.411462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:26:01.411462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 10:26:01.411462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:01.411462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 476/958] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 18 10:26:01 2022] POST /identity/v3/users => generated 160 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:01.412471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a08754-df22-422b-9c4f-f5d849676ba9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:01.535826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a08754-df22-422b-9c4f-f5d849676ba9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:01.548806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:01.577546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9019b-7d7b-4385-a540-6822905f628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:01.578373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9019b-7d7b-4385-a540-6822905f628d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:01.578656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9019b-7d7b-4385-a540-6822905f628d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:01.578934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9019b-7d7b-4385-a540-6822905f628d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:01.587257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9019b-7d7b-4385-a540-6822905f628d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:01.588745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9019b-7d7b-4385-a540-6822905f628d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-06f9019b-7d7b-4385-a540-6822905f628d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:01.604562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:01.607942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:01.610214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 483/959] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:01.617978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:01.725166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5b9cd-6ea3-4989-bd72-f3b41b1d5b28 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:01.762926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc9688-6d7b-4a58-8280-9ce0b08da618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:01.762926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc9688-6d7b-4a58-8280-9ce0b08da618 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:01.762926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc9688-6d7b-4a58-8280-9ce0b08da618 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:01.763299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc9688-6d7b-4a58-8280-9ce0b08da618 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:01.764499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc9688-6d7b-4a58-8280-9ce0b08da618 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:01.764499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc9688-6d7b-4a58-8280-9ce0b08da618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:01.767941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc9688-6d7b-4a58-8280-9ce0b08da618 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:01.812864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 477/960] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 18 10:26:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:01.904313 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51fc48-2272-454a-b0e7-0385e7801cc6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:01.904691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51fc48-2272-454a-b0e7-0385e7801cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:01.904918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51fc48-2272-454a-b0e7-0385e7801cc6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:01.906721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 484/961] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:01.912934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e6d6-1403-4fca-a82e-159e41134a55 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:01.913252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e6d6-1403-4fca-a82e-159e41134a55 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:01.913505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e6d6-1403-4fca-a82e-159e41134a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:01.932494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51a6e6d6-1403-4fca-a82e-159e41134a55 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:01.934882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51a6e6d6-1403-4fca-a82e-159e41134a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:01.991664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:02.015082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 478/962] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:02.038680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51fc48-2272-454a-b0e7-0385e7801cc6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:02.152067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51fc48-2272-454a-b0e7-0385e7801cc6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:02.189906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:02.190714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:02.190998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:02.191275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:02.191937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:02.193353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:02.206401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:02.207593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:02.208437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 485/963] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:02.213635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6e6d6-1403-4fca-a82e-159e41134a55 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:02.334269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6e6d6-1403-4fca-a82e-159e41134a55 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:02.371242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fb7ac-7a95-4801-9554-aea4eddc793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:02.372340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb7ac-7a95-4801-9554-aea4eddc793c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:02.372622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb7ac-7a95-4801-9554-aea4eddc793c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:02.372925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb7ac-7a95-4801-9554-aea4eddc793c None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:02.373638 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fb7ac-7a95-4801-9554-aea4eddc793c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:02.373945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fb7ac-7a95-4801-9554-aea4eddc793c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:02.375325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fb7ac-7a95-4801-9554-aea4eddc793c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:02.380398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 479/964] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:26:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:02.386007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:02.417783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:02.493214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75c586-c22e-42e6-9655-118385ce8e56 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:02.528277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f9422-7d69-4faa-89f3-4c75022c5799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:02.528976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f9422-7d69-4faa-89f3-4c75022c5799 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:02.529262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f9422-7d69-4faa-89f3-4c75022c5799 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:02.529535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f9422-7d69-4faa-89f3-4c75022c5799 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:02.530305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f9422-7d69-4faa-89f3-4c75022c5799 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:02.530618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f9422-7d69-4faa-89f3-4c75022c5799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:02.532090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f9422-7d69-4faa-89f3-4c75022c5799 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:02.537059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 486/965] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:26:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:02.784357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946333d-9ded-4d80-abe0-76153e172f10 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:02.784767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946333d-9ded-4d80-abe0-76153e172f10 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:02.784968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946333d-9ded-4d80-abe0-76153e172f10 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:02.786468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 480/966] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:02.791903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285a499-1504-49fe-a9ad-daca4b579307 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:02.792197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285a499-1504-49fe-a9ad-daca4b579307 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:02.792480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285a499-1504-49fe-a9ad-daca4b579307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:02.810118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a285a499-1504-49fe-a9ad-daca4b579307 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:02.811307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a285a499-1504-49fe-a9ad-daca4b579307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:02.882215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 487/967] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:02.906065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946333d-9ded-4d80-abe0-76153e172f10 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:02.956133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.016610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946333d-9ded-4d80-abe0-76153e172f10 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.053762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0987-3fc2-4057-87b2-52dc753f3ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.054496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0987-3fc2-4057-87b2-52dc753f3ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.054775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0987-3fc2-4057-87b2-52dc753f3ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.055047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0987-3fc2-4057-87b2-52dc753f3ab4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.065846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec0987-3fc2-4057-87b2-52dc753f3ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c9ec0987-3fc2-4057-87b2-52dc753f3ab4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:03.074565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:03.075539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:03.076405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:03.083996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 481/968] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 18 10:26:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:03.084151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285a499-1504-49fe-a9ad-daca4b579307 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.109675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.111078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af13dd-e067-430b-a920-ca216ad7a4e3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.111252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af13dd-e067-430b-a920-ca216ad7a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.111509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af13dd-e067-430b-a920-ca216ad7a4e3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.113076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 482/969] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:03.118242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f8fe0-e193-449a-ad8a-78bac65f612a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.118527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f8fe0-e193-449a-ad8a-78bac65f612a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.124214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f8fe0-e193-449a-ad8a-78bac65f612a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.136116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c5f8fe0-e193-449a-ad8a-78bac65f612a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:03.138038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c5f8fe0-e193-449a-ad8a-78bac65f612a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:03.196187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285a499-1504-49fe-a9ad-daca4b579307 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.220830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 483/970] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:03.232896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.233592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.233876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.234146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.234889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:03.235195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.236559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.243092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f8fe0-e193-449a-ad8a-78bac65f612a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.245716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 488/971] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:03.256825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.298180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.365258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.384109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f8fe0-e193-449a-ad8a-78bac65f612a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.390578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdde07-42e1-4aa3-ac29-32cfecabde23 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.422787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.423604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.423932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.424211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.424893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.426703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:03.439034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:03.440038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:03.440932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 484/972] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:03.446060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.450175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.451524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.452020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.452513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.471205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.473934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:03.505474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:03.506452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:03.507373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:03.507373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:03.507373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:03.507373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:03.513204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 489/973] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:03.516826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.567765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.571261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963e534-3a86-4f84-9b07-22c0051cf7f3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.608522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.609211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.609498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.609778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.610494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:03.610804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.612191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.620923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 485/974] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:03.623228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.634335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d61c-1df7-49d1-bbf2-9ebac2512099 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.675223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.676069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.676347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.676623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.677388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:03.677696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.679063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.688109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.693491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 490/975] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:03.699598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.745173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17e9cb-7aa5-40d9-ad57-eeaee46141c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.787999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.788680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.789019 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.789293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.790015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:03.790323 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.791766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.804247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 486/976] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:03.809960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.824285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b678-3d49-44ea-bfd0-14303691bb23 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.842769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.864705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.865569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.865859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.866137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.874365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.875749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:03.891316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:03.892465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:03.894549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 491/977] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:03.898470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:03.930962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7c1c7-971a-4a13-b88a-5e609010b6e7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:03.962578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:03.971795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:03.972591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:03.972887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:03.973250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:03.973979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] RBAC: Adding query filter params (interface=admin, service_id=245b2e848318421497f55fce9362103c, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:03.974298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:03.975713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:03.982159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 487/978] 10.222.0.49 () {56 vars in 1249 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/endpoints?service_id=245b2e848318421497f55fce9362103c&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:03.987997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:04.012572 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fa3ec-f436-4e31-97b3-8f99ee08cebc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:04.048947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:04.049628 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.049907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.050184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.051027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:04.051336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:04.052772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:04.077751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 492/979] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:04.078795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:04.097629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:04.108162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a169d-df75-4556-b605-7c1f4e2b76a4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:04.147583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:04.148171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.148618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.148943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.150156 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:04.151812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:04.165643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 488/980] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:26:03 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:26:04.188898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac436401-c59b-4f43-8dfc-06c56d2a898b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:04.209646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:04.229924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:04.231539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.231539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.231792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.307503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:04.309219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:04.416172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 493/981] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:04 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:04.422343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:04.527446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba86e10-70fe-4d6e-aab0-fa90aa97a913 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:04.563324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04eb0a-6c33-49cf-bd19-166f66ebf026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:04.564119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04eb0a-6c33-49cf-bd19-166f66ebf026 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.564402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04eb0a-6c33-49cf-bd19-166f66ebf026 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.564675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04eb0a-6c33-49cf-bd19-166f66ebf026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.575332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04eb0a-6c33-49cf-bd19-166f66ebf026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:04.576897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04eb0a-6c33-49cf-bd19-166f66ebf026 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5d04eb0a-6c33-49cf-bd19-166f66ebf026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:04.594837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:04.595790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:04.596706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:04.596706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:04.596706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:04.596706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:04.596706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 489/982] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:04.603037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:04.623740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:04.711796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031fe76-d1f0-4c4f-8ad0-c8d409585e9a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:04.726879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e8b4e-45c2-4d33-9c9d-4ef054230c50 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.727273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e8b4e-45c2-4d33-9c9d-4ef054230c50 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.727634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e8b4e-45c2-4d33-9c9d-4ef054230c50 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.729155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 490/983] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:04.734547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce75e1-e42a-49b9-ba85-c1818a7d0a7e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.734834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce75e1-e42a-49b9-ba85-c1818a7d0a7e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.735112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce75e1-e42a-49b9-ba85-c1818a7d0a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.753084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0bce75e1-e42a-49b9-ba85-c1818a7d0a7e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:04.754361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0bce75e1-e42a-49b9-ba85-c1818a7d0a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:04.755349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:04.756021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.756311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.756556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:04.756654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.757410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:04.757724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:04.759138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:04.771757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 494/984] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:04.777923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:04.832852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 491/985] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:04.856941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce75e1-e42a-49b9-ba85-c1818a7d0a7e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:04.865723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:04.891920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8fab1-a9bf-49dd-a274-496173b1b628 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:04.932441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:04.933824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.934158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.935083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.944556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:04.946358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:04.961024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce75e1-e42a-49b9-ba85-c1818a7d0a7e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:04.966317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:04.967327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:04.968282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:04.969612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 495/986] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:04.973176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:05.003848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:05.004521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:05.004818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:05.005104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:05.005835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:05.007227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:05.081639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:05.089052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 492/987] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:26:04 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:26:05.094975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dee7e3-fc94-4b08-8219-86d502403035 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:05.142049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:05.148392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2965c8-d8a0-45bf-abd5-365239aa207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:05.149397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2965c8-d8a0-45bf-abd5-365239aa207c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:05.149863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2965c8-d8a0-45bf-abd5-365239aa207c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:05.150289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2965c8-d8a0-45bf-abd5-365239aa207c None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:05.151255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2965c8-d8a0-45bf-abd5-365239aa207c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:05.151722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2965c8-d8a0-45bf-abd5-365239aa207c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:05.153413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2965c8-d8a0-45bf-abd5-365239aa207c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:05.166633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 496/988] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:05.174232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:05.297981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e497dee-f313-4e2e-89bd-f9eb3b74f9d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:05.333574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbf39e-8389-43a0-8729-14d4157f1b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:05.334261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbf39e-8389-43a0-8729-14d4157f1b84 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:05.334634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbf39e-8389-43a0-8729-14d4157f1b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:05.334914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbf39e-8389-43a0-8729-14d4157f1b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:05.335773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbf39e-8389-43a0-8729-14d4157f1b84 None admin] RBAC: Adding query filter params (role.id=5c9ff5726a4f4a3c86b6504b08ec2239, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=41cc665cbec74e06bfa7acf97f968e52) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:05.336091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbf39e-8389-43a0-8729-14d4157f1b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:05.337623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbf39e-8389-43a0-8729-14d4157f1b84 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:05.344922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 493/989] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:05 2022] GET /identity/v3/role_assignments?user.id=41cc665cbec74e06bfa7acf97f968e52&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=5c9ff5726a4f4a3c86b6504b08ec2239 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:06.305984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e818f-8fc5-4356-a0ed-bc4548df4642 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:06.306520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e818f-8fc5-4356-a0ed-bc4548df4642 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:06.306520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e818f-8fc5-4356-a0ed-bc4548df4642 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:06.308804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 497/990] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:06.314353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff47b-324f-467f-a165-285f1d317bbf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:06.314871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff47b-324f-467f-a165-285f1d317bbf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:06.315148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff47b-324f-467f-a165-285f1d317bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:06.332650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e93ff47b-324f-467f-a165-285f1d317bbf None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:06.333880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e93ff47b-324f-467f-a165-285f1d317bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:06.397113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:06.409769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 494/991] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:06.445873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e818f-8fc5-4356-a0ed-bc4548df4642 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:06.588410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e818f-8fc5-4356-a0ed-bc4548df4642 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:06.625062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:06.626194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:06.626620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:06.627084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:06.627974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:06.629573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:06.642919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:06.644068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:26:06.644939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:06.647302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 498/992] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 18 10:26:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:06.657052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ff47b-324f-467f-a165-285f1d317bbf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:06.765245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ff47b-324f-467f-a165-285f1d317bbf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:06.808582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:06.809640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:06.809948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:06.810224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:06.810942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:06.811253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:06.812692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:06.822536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 495/993] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:26:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:06.824520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:06.887234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:06.935526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c7d92-b107-440f-a372-f08f043fce10 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:06.973623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cb43a-7761-4278-98a2-d25bbe0f8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:06.974297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb43a-7761-4278-98a2-d25bbe0f8ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:06.974585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb43a-7761-4278-98a2-d25bbe0f8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:06.974860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb43a-7761-4278-98a2-d25bbe0f8ac9 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:06.975581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb43a-7761-4278-98a2-d25bbe0f8ac9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:06.975890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb43a-7761-4278-98a2-d25bbe0f8ac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:06.977312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb43a-7761-4278-98a2-d25bbe0f8ac9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:06.982632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 499/994] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 18 10:26:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:06.988180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.049214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:07.097417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79603457-a87b-4ebf-9490-87e1b9582ed1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.133757 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.134422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.134707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.134985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.135746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:07.137915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:07.172906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa728cd5-8995-4d45-8f79-bf37261a3638 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.173203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa728cd5-8995-4d45-8f79-bf37261a3638 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.173488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa728cd5-8995-4d45-8f79-bf37261a3638 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.174867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 500/995] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:07.179996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae3966-43b1-46b0-b912-0c43e963fdff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.180286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae3966-43b1-46b0-b912-0c43e963fdff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.180570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae3966-43b1-46b0-b912-0c43e963fdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.197803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-72ae3966-43b1-46b0-b912-0c43e963fdff None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:07.199772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-72ae3966-43b1-46b0-b912-0c43e963fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:07.208850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 496/996] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:26:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:07.297796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 501/997] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:07.321870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.445132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fe0e-ba52-4967-8a47-6bc2c0dcd52a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.462564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:07.481332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.483183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6a3ff-9f17-45ba-a39a-d791776398a6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.483671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6a3ff-9f17-45ba-a39a-d791776398a6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.483770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6a3ff-9f17-45ba-a39a-d791776398a6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.484368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.484663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.484972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.485395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 502/998] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:07.485648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:07.487099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:07.492708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5780-c918-46e8-afa8-fc3bb66ca2e6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.493021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5780-c918-46e8-afa8-fc3bb66ca2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.493302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5780-c918-46e8-afa8-fc3bb66ca2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:07.501985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:07.503082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:07.504029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:07.505400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 497/999] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:07.511932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.518604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dadf5780-c918-46e8-afa8-fc3bb66ca2e6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:07.519934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dadf5780-c918-46e8-afa8-fc3bb66ca2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:07.569299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:07.597099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 503/1000] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:07.619704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf5780-c918-46e8-afa8-fc3bb66ca2e6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.624134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdafd4-2471-4f4f-a225-04bc497d6d10 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.662840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.663521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.663809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.664088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.664837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:07.665153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:07.666575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:07.672674 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 498/1001] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:07.678448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.732204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:07.742729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf5780-c918-46e8-afa8-fc3bb66ca2e6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.783283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.784109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.784395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.784675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.785372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:07.786790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:07.791858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc264-32f2-4b39-9c1b-2fd50d1ed0cc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:07.800706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:07.807471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:07.808996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 504/1002] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:07.811471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.836543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.837307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.837581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.837861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.838566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:07.838870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:07.840624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:07.850489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 499/1003] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:07.851528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.864309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:07.936326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6786e-cb89-4110-89e0-19975cb3f5e2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.967694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427924ac-06af-4249-98a6-4b6ef7349393 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.973745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.974737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.975183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.975644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.976517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] RBAC: Adding query filter params (name=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:07.977074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:07.977855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:07.979405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:07.990587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 505/1004] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:07.991737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.006387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.007050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.007338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.007617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.008422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] RBAC: Adding query filter params (interface=public, service_id=f2da808dace149159610e3c61c6326f5, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:08.008737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.010226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.018033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 500/1005] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/endpoints?service_id=f2da808dace149159610e3c61c6326f5&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:08.023368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.110497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16316-3a1d-45d3-8411-7bf1b172789b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:08.113279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:08.137918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cead58f-9c15-4e97-b4cb-cc48f759eaf8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:08.146818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8252e-13fc-499d-aca5-5b5eb1e64252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.147496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8252e-13fc-499d-aca5-5b5eb1e64252 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.147781 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8252e-13fc-499d-aca5-5b5eb1e64252 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.148163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8252e-13fc-499d-aca5-5b5eb1e64252 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.148911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8252e-13fc-499d-aca5-5b5eb1e64252 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:08.149219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8252e-13fc-499d-aca5-5b5eb1e64252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.150658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8252e-13fc-499d-aca5-5b5eb1e64252 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.156149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 506/1006] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:08.177453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc842e-1e8c-42f4-9cf0-530b42f6daef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.178226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc842e-1e8c-42f4-9cf0-530b42f6daef None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.178530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc842e-1e8c-42f4-9cf0-530b42f6daef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.178833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc842e-1e8c-42f4-9cf0-530b42f6daef None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.179606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc842e-1e8c-42f4-9cf0-530b42f6daef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.181147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc842e-1e8c-42f4-9cf0-530b42f6daef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.197104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 501/1007] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:26:08 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:26:09.287582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856c3d-329c-4bd3-86c6-3896411bce9a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.288133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856c3d-329c-4bd3-86c6-3896411bce9a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.288133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856c3d-329c-4bd3-86c6-3896411bce9a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.289708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 507/1008] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:09.295717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc81c1-7d81-4b4e-93c2-c9975a57ee9c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.296197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc81c1-7d81-4b4e-93c2-c9975a57ee9c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.296292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc81c1-7d81-4b4e-93c2-c9975a57ee9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.313150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1fdc81c1-7d81-4b4e-93c2-c9975a57ee9c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:09.314214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fdc81c1-7d81-4b4e-93c2-c9975a57ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:09.348908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:09.388207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 502/1009] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:09.411176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856c3d-329c-4bd3-86c6-3896411bce9a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:09.518263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856c3d-329c-4bd3-86c6-3896411bce9a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:09.552466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9f1a4-7eda-49fc-b847-f45536cdedbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:09.553167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9f1a4-7eda-49fc-b847-f45536cdedbe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.553458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9f1a4-7eda-49fc-b847-f45536cdedbe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.553728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9f1a4-7eda-49fc-b847-f45536cdedbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.554967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9f1a4-7eda-49fc-b847-f45536cdedbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:09.556391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9f1a4-7eda-49fc-b847-f45536cdedbe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:09.558302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 508/1010] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:09.564020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc81c1-7d81-4b4e-93c2-c9975a57ee9c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:09.666432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc81c1-7d81-4b4e-93c2-c9975a57ee9c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:09.701912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfafca-e442-4dd2-982e-643f0b2d038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:09.702577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfafca-e442-4dd2-982e-643f0b2d038e None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.702862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfafca-e442-4dd2-982e-643f0b2d038e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.703140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfafca-e442-4dd2-982e-643f0b2d038e None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.703869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfafca-e442-4dd2-982e-643f0b2d038e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:09.705353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfafca-e442-4dd2-982e-643f0b2d038e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:09.845080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 503/1011] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:26:09 2022] POST /identity/v3/projects => generated 311 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:26:10.209341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aec56-1c78-4eb6-a1f2-4098c5a09146 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.209694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aec56-1c78-4eb6-a1f2-4098c5a09146 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.209917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aec56-1c78-4eb6-a1f2-4098c5a09146 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.211459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 509/1012] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:10.217365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e75a3c-2bd4-4ce3-b8cf-043dbe145abb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.217756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e75a3c-2bd4-4ce3-b8cf-043dbe145abb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.218133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e75a3c-2bd4-4ce3-b8cf-043dbe145abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.235619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a0e75a3c-2bd4-4ce3-b8cf-043dbe145abb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:10.236839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0e75a3c-2bd4-4ce3-b8cf-043dbe145abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:10.251486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:10.276398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a26d-efac-4d9d-aec9-5bfb942593dc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.277866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a26d-efac-4d9d-aec9-5bfb942593dc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.277866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a26d-efac-4d9d-aec9-5bfb942593dc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.279103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 510/1013] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:10.284479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b17590-43d6-4234-8e91-9e25a37d21aa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.284763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b17590-43d6-4234-8e91-9e25a37d21aa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.285092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b17590-43d6-4234-8e91-9e25a37d21aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.303931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-20b17590-43d6-4234-8e91-9e25a37d21aa None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:10.305191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-20b17590-43d6-4234-8e91-9e25a37d21aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:10.313733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 504/1014] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:10.337158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e75a3c-2bd4-4ce3-b8cf-043dbe145abb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:10.341457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:10.396830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 511/1015] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:10.412380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b17590-43d6-4234-8e91-9e25a37d21aa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:10.456297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e75a3c-2bd4-4ce3-b8cf-043dbe145abb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:10.500054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:10.500638 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.500960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.501242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.501980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:10.503781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:10.529186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b17590-43d6-4234-8e91-9e25a37d21aa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:10.574745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:10.575564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.575946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.576235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.576940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:10.578477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:10.588651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 505/1016] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:26:10 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:10.593843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:10.595249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:26:10.596316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:10.600600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 512/1017] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:10.602964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:10.617162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:10.735738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06268a9b-feff-45f4-8bf2-4055245f329d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:10.773819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:10.774413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.774697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.774978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.775703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:10.776016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:10.777998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:10.788741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 506/1018] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:26:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:10.790106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:10.831503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:10.908554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826b07f-95f8-4585-9fe3-4b1540522007 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:10.959804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508be076-a726-4b2b-a99c-fbeee8ee9a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:10.960478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be076-a726-4b2b-a99c-fbeee8ee9a95 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:10.960764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be076-a726-4b2b-a99c-fbeee8ee9a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:10.961067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be076-a726-4b2b-a99c-fbeee8ee9a95 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:10.961795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be076-a726-4b2b-a99c-fbeee8ee9a95 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:10.962102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be076-a726-4b2b-a99c-fbeee8ee9a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:10.963489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be076-a726-4b2b-a99c-fbeee8ee9a95 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:10.973436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 513/1019] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 18 10:26:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:10.974657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:11.049280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:11.083784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad84bfc-96fa-45be-8adc-73b614737783 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:11.123824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1ec75-c062-4dee-bd5e-a58e9ef72ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:11.124584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1ec75-c062-4dee-bd5e-a58e9ef72ace None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:11.124972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1ec75-c062-4dee-bd5e-a58e9ef72ace None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:11.125316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1ec75-c062-4dee-bd5e-a58e9ef72ace None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:11.126149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1ec75-c062-4dee-bd5e-a58e9ef72ace None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:11.127601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1ec75-c062-4dee-bd5e-a58e9ef72ace None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:11.244821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 507/1020] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:26:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:26:11.999082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6076c1-3335-4f18-bd32-5d79ffd679d8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:11.999663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6076c1-3335-4f18-bd32-5d79ffd679d8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:11.999663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6076c1-3335-4f18-bd32-5d79ffd679d8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.001059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 514/1021] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:12.006224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76e42-5c3b-4d6d-a173-0bfc49613013 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.006625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76e42-5c3b-4d6d-a173-0bfc49613013 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.006820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76e42-5c3b-4d6d-a173-0bfc49613013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.024478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-efb76e42-5c3b-4d6d-a173-0bfc49613013 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:12.025689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-efb76e42-5c3b-4d6d-a173-0bfc49613013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:12.108906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 508/1022] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:12.129951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6076c1-3335-4f18-bd32-5d79ffd679d8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.229014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:12.237812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6076c1-3335-4f18-bd32-5d79ffd679d8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.273746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb5cb8-761b-42e4-89dc-5eacd4909658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.274544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb5cb8-761b-42e4-89dc-5eacd4909658 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.274827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb5cb8-761b-42e4-89dc-5eacd4909658 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.275100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb5cb8-761b-42e4-89dc-5eacd4909658 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.277999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb5cb8-761b-42e4-89dc-5eacd4909658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:12.279735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb5cb8-761b-42e4-89dc-5eacd4909658 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:12.286557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 515/1023] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:12.287590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76e42-5c3b-4d6d-a173-0bfc49613013 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.341058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:12.394058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76e42-5c3b-4d6d-a173-0bfc49613013 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.435760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307eac4-8d1b-408b-b18a-5b4a112b8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.436633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307eac4-8d1b-408b-b18a-5b4a112b8a97 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.437043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307eac4-8d1b-408b-b18a-5b4a112b8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.437411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307eac4-8d1b-408b-b18a-5b4a112b8a97 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.438386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307eac4-8d1b-408b-b18a-5b4a112b8a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:12.440344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307eac4-8d1b-408b-b18a-5b4a112b8a97 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:12.557375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 509/1024] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:26:12 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:26:12.770646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a20e-2661-4dad-a0cd-515bc6052546 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.770990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a20e-2661-4dad-a0cd-515bc6052546 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.771334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a20e-2661-4dad-a0cd-515bc6052546 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.772719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 516/1025] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:12.778976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d85e-9136-44bc-85ba-9165cb219d39 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.779266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d85e-9136-44bc-85ba-9165cb219d39 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.779544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d85e-9136-44bc-85ba-9165cb219d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.797742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1e39d85e-9136-44bc-85ba-9165cb219d39 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:12.799112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1e39d85e-9136-44bc-85ba-9165cb219d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:12.874759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 510/1026] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:12.897528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a20e-2661-4dad-a0cd-515bc6052546 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.011732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a20e-2661-4dad-a0cd-515bc6052546 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.048422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.049161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.049456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.049728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] PATH_INFO: `/v3/services/image` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.050387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.051764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:13.064351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:13.065643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:13.066805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 517/1027] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:13.075062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39d85e-9136-44bc-85ba-9165cb219d39 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.115497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.187562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39d85e-9136-44bc-85ba-9165cb219d39 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.225130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.225798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.226083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.226364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.227070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:13.227376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.228769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.239179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 511/1028] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:13.240416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.269664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.347497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eeb8f-bf56-4d74-8069-56dc8755903c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.383266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564f5fb-daa5-4ffd-a1e4-782e07f66699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.383933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f5fb-daa5-4ffd-a1e4-782e07f66699 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.384216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f5fb-daa5-4ffd-a1e4-782e07f66699 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.384490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f5fb-daa5-4ffd-a1e4-782e07f66699 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.385229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f5fb-daa5-4ffd-a1e4-782e07f66699 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:13.385538 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f5fb-daa5-4ffd-a1e4-782e07f66699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.386946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f5fb-daa5-4ffd-a1e4-782e07f66699 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.392197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 518/1029] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:13.398062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.429575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de20a-bce6-46b1-8294-c85a544c977c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.429868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de20a-bce6-46b1-8294-c85a544c977c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.430150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de20a-bce6-46b1-8294-c85a544c977c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.431576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 519/1030] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:13.437019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15203aa5-dd29-4f39-8684-5d63c546980f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.437307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15203aa5-dd29-4f39-8684-5d63c546980f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.437584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15203aa5-dd29-4f39-8684-5d63c546980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.454767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-15203aa5-dd29-4f39-8684-5d63c546980f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:13.455966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-15203aa5-dd29-4f39-8684-5d63c546980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:13.506772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.510430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb2ff3-af10-4e8f-841d-2ed17825d731 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.525970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 520/1031] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:13.545305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.545968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.546251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.546527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.547319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] RBAC: Adding query filter params (interface=public, service_id=9c557469e28a432996680bf79da2e18a, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:13.547637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.549088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.549692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15203aa5-dd29-4f39-8684-5d63c546980f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.561407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 512/1032] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/endpoints?service_id=9c557469e28a432996680bf79da2e18a&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:13.562611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.576975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.673682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15203aa5-dd29-4f39-8684-5d63c546980f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.673882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e9cc1-9f9c-44f4-8cc2-2887ea74cf24 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.710350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.713471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.713471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.713471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.713471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.713891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.717094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.717965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.718249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.718520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.721248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 513/1033] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:26:13.734357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:13.743112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:13.744453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:13.746490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 521/1034] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:13.752190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.794176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.883966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995d9e6-0dc0-41e2-8b08-4be9fb607e4e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.923373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.924104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.924332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.924602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.925432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:13.925738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.927132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.937930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 514/1035] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:13.942924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:14.030945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:14.052473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480f169-a8f5-4240-8644-e0b944c906a7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:14.089595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:14.090407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.090696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.090971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.102388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:14.103925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:14.124578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:14.125778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:14.126696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:14.126696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:14.126696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:14.126696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:14.130671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 522/1036] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:14.132053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:14.232590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:14.242683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd2fda-d94f-4310-b291-e954aa5ca5a2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:14.279422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:14.280094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.280381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.280656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.281438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:14.281774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:14.283191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:14.301009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 515/1037] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:14.302177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:14.408304 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf253f-732a-4df8-bc7e-24e0db97c91c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:14.441751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:14.447394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:14.449096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.449096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.449524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.457970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:14.459524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:14.475095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:14.476408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:14.477887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 523/1038] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:26:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:14.485246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:14.593492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c155-2e3d-47e5-8f4f-327a81b17bd9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:14.629877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565c36-d318-4664-9af8-3a9c41a225b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:14.630601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565c36-d318-4664-9af8-3a9c41a225b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.630843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565c36-d318-4664-9af8-3a9c41a225b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.631169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565c36-d318-4664-9af8-3a9c41a225b3 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.632017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565c36-d318-4664-9af8-3a9c41a225b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:14.632330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565c36-d318-4664-9af8-3a9c41a225b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:14.633821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565c36-d318-4664-9af8-3a9c41a225b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:14.654128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 516/1039] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:26:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:14.659876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:14.679037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea43c3-4fc1-467f-a617-31c963a03f8c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.679037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea43c3-4fc1-467f-a617-31c963a03f8c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.679037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea43c3-4fc1-467f-a617-31c963a03f8c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.679830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 517/1040] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:14.684616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695299-4c40-48a3-9c4f-a3e300ec8429 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.684943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695299-4c40-48a3-9c4f-a3e300ec8429 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.685224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695299-4c40-48a3-9c4f-a3e300ec8429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.701601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bb695299-4c40-48a3-9c4f-a3e300ec8429 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:14.702786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb695299-4c40-48a3-9c4f-a3e300ec8429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:14.705654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:14.770536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f05ae6-e614-485d-89fa-ca99e8ac91d4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:14.784551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 518/1041] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:14.807505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695299-4c40-48a3-9c4f-a3e300ec8429 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:14.817961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:14.818630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.819254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.819547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.892883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:14.905792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:14.907736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:14.921812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695299-4c40-48a3-9c4f-a3e300ec8429 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:14.964752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:14.965580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:14.965863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:14.966426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:14.973407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:14.977438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:14.985366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:14.986573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:14.987560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 519/1042] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:14.993240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.028885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 524/1043] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:14 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:26:15.034592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.111467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a59a0-6257-41f7-ab31-34e8e4a061d5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.135156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.146553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a38b-008c-47be-9a27-c8eec4be0b5f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.154846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.154846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.154846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.154846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.155533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:15.155533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.155533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.163482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 520/1044] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:15.169684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.185208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.186015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.186303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.186582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.197500 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.199644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:15.218024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:15.219195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:15.220125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:15.220125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:15.220125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:15.220125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:15.220125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 525/1045] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:15.224991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.244330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.290286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f26b1-046a-4402-b9f8-f1f7b652794e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.325626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.326507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.326792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.327074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] PATH_INFO: `/v3/users/c186f06e18a14abdaa6c15d9eec3f1e4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.334916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8a099-cae0-4b93-89c2-af4b93905c77 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.342243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] RBAC: Authorizing `identity:get_user(user_id=c186f06e18a14abdaa6c15d9eec3f1e4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.343711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.345466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 521/1046] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/users/c186f06e18a14abdaa6c15d9eec3f1e4 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:15.352894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.379240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.379970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.382043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.382043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.382043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:15.382043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.383215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.396011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 526/1047] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:15.402594 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.409028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.478412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289bd96-6163-4bdf-bf26-00fc467de5ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.499247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.516989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.517642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.517918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.518191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.522504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448e410-f867-4747-a5c1-99f1633638b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.560714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.561475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.561761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.562037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.569165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.570615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:15.588062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:15.589651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:15.590669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:15.596541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 527/1048] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:15.597838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.599492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.601052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.664917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.708303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fb6e4-6432-40c5-a23f-8e74e6b7a163 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.711813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 522/1049] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:26:15.718236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.738145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.754113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.754882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.755233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.756276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.756398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:15.756827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.758326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.772683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 528/1050] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:15.779960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.829292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79619314-0b8f-4c18-8cae-f3a11e98b214 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.868381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e79cfe-d107-4283-b885-c22f6506cbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.869486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e79cfe-d107-4283-b885-c22f6506cbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.869768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e79cfe-d107-4283-b885-c22f6506cbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.870307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e79cfe-d107-4283-b885-c22f6506cbc8 None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.872038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e79cfe-d107-4283-b885-c22f6506cbc8 None admin] RBAC: Authorizing `identity:get_project(project_id=b3401968aa89436194cb7f345df1d38c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.873520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e79cfe-d107-4283-b885-c22f6506cbc8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.875221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 523/1051] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/projects/b3401968aa89436194cb7f345df1d38c => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:15.876506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1388f3-dfb7-4776-a9b7-c6a42fbdfab3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.876800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1388f3-dfb7-4776-a9b7-c6a42fbdfab3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.877072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1388f3-dfb7-4776-a9b7-c6a42fbdfab3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.882237 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 524/1052] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:15.882237 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1388f3-dfb7-4776-a9b7-c6a42fbdfab3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.895367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6d5e8-423b-4a68-a58c-452303ebc8dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:15.921933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.933367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944fb67-a084-4ba2-80ce-b010365a419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:15.934085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944fb67-a084-4ba2-80ce-b010365a419b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.939191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944fb67-a084-4ba2-80ce-b010365a419b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.939191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944fb67-a084-4ba2-80ce-b010365a419b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.939191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944fb67-a084-4ba2-80ce-b010365a419b None admin] RBAC: Adding query filter params (role.id=5c9ff5726a4f4a3c86b6504b08ec2239, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=ec899ae414744dde9ec9b76aac1ce9d8) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:15.939191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944fb67-a084-4ba2-80ce-b010365a419b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:15.939191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944fb67-a084-4ba2-80ce-b010365a419b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:15.942771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 529/1053] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/role_assignments?user.id=ec899ae414744dde9ec9b76aac1ce9d8&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=5c9ff5726a4f4a3c86b6504b08ec2239 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:15.944019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae13f94-8c52-4ca2-9eef-e84b36c8ecb2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.944312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae13f94-8c52-4ca2-9eef-e84b36c8ecb2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.944591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae13f94-8c52-4ca2-9eef-e84b36c8ecb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.962529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4ae13f94-8c52-4ca2-9eef-e84b36c8ecb2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:15.964408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ae13f94-8c52-4ca2-9eef-e84b36c8ecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:15.978028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:16.000360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1388f3-dfb7-4776-a9b7-c6a42fbdfab3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.044204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.044944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.045202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.045484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.046408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=b3401968aa89436194cb7f345df1d38c, user.id=c186f06e18a14abdaa6c15d9eec3f1e4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:16.046730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.048269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.049643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 530/1054] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:16.063406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 525/1055] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/role_assignments?user.id=c186f06e18a14abdaa6c15d9eec3f1e4&scope.project.id=b3401968aa89436194cb7f345df1d38c&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:16.089372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae13f94-8c52-4ca2-9eef-e84b36c8ecb2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.211764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae13f94-8c52-4ca2-9eef-e84b36c8ecb2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.233839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:16.251375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.252187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.252469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.252744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.253504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.254887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:16.267195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:16.268228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:16.269110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 531/1056] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:16.277486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.381936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4ed60-51d5-4376-992c-ee8876d635eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.419756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.420432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.420716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.421020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.421731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:16.422039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.423424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.433565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 526/1057] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:16.434961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.545987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f962e5-5259-4f8a-9f3e-56c27a5f35e8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.581327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47400cf4-ed25-48ee-8bd0-9c02cb0eba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.581998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400cf4-ed25-48ee-8bd0-9c02cb0eba2b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.582279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400cf4-ed25-48ee-8bd0-9c02cb0eba2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.582550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400cf4-ed25-48ee-8bd0-9c02cb0eba2b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.583258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47400cf4-ed25-48ee-8bd0-9c02cb0eba2b None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:16.583648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47400cf4-ed25-48ee-8bd0-9c02cb0eba2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.585130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47400cf4-ed25-48ee-8bd0-9c02cb0eba2b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.598015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 532/1058] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:16.599441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.617133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:16.705182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95857bf-5dd2-42fa-86bb-28a0e8641ba1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.740007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c73a7c-3313-4fc1-be6e-4d6d5aea5510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.740673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c73a7c-3313-4fc1-be6e-4d6d5aea5510 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.740986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c73a7c-3313-4fc1-be6e-4d6d5aea5510 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.741262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c73a7c-3313-4fc1-be6e-4d6d5aea5510 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.742008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c73a7c-3313-4fc1-be6e-4d6d5aea5510 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.743415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c73a7c-3313-4fc1-be6e-4d6d5aea5510 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.818718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 527/1059] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:26:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:18.040239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e3652-9b2c-4f49-b26b-230807705d83 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.040976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e3652-9b2c-4f49-b26b-230807705d83 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.040976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e3652-9b2c-4f49-b26b-230807705d83 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.043262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 533/1060] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:18.048645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982db09-d8ed-4976-bdc1-a2124b7a71aa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.049132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982db09-d8ed-4976-bdc1-a2124b7a71aa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.049333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982db09-d8ed-4976-bdc1-a2124b7a71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.067099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4982db09-d8ed-4976-bdc1-a2124b7a71aa None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:18.068220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4982db09-d8ed-4976-bdc1-a2124b7a71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:18.104062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ea518-3770-49ce-b44f-09026f187d33 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.104618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ea518-3770-49ce-b44f-09026f187d33 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.104755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ea518-3770-49ce-b44f-09026f187d33 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.106204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 534/1061] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:18.111884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c35db-d30a-4b3a-9091-d8bac3bba26a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.112184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c35db-d30a-4b3a-9091-d8bac3bba26a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.112466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c35db-d30a-4b3a-9091-d8bac3bba26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.129422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9e7c35db-d30a-4b3a-9091-d8bac3bba26a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:18.130608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9e7c35db-d30a-4b3a-9091-d8bac3bba26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:18.141461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:18.153878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 528/1062] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:18.177766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982db09-d8ed-4976-bdc1-a2124b7a71aa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.187368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:18.220930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 535/1063] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:18.238484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c35db-d30a-4b3a-9091-d8bac3bba26a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.289373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982db09-d8ed-4976-bdc1-a2124b7a71aa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.329510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.330256 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.330538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.330816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.343477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:18.347580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c35db-d30a-4b3a-9091-d8bac3bba26a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:18.351665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:18.352696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:18.353601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 529/1064] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:18.359809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.391183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.391981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.392266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.392539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.406009 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:18.413691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:18.414904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:18.415916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 536/1065] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:18.421936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.435380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:18.465307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:18.486032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8cde5-1570-496c-828a-66dabe7849d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.523914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.524585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.524898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.525180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.525957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:18.526796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:18.528387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:18.535052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711116e8-2290-4aa9-9bc3-8ab9b2f850c8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.536739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 530/1066] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:18.543321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.576990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.577667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.578023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.578237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.579025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:18.579389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:18.580845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:18.593341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 537/1067] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:18.599087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.654488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265969-3e77-43f7-a88e-06a18c35708e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.673803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:18.692811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.693728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.694105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.694447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.701799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:18.705738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753e850-a842-4f3e-85d2-9a2e534467f9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.707703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:18.709328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:18.728010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:18.735001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:18.736103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:18.736103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:18.736103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:18.736103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:18.736103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 531/1068] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:18.737898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.746943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.748303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.748689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.749117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] PATH_INFO: `/v3/users/c186f06e18a14abdaa6c15d9eec3f1e4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.758524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] RBAC: Authorizing `identity:get_user(user_id=c186f06e18a14abdaa6c15d9eec3f1e4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:18.758524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:18.758524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 538/1069] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/users/c186f06e18a14abdaa6c15d9eec3f1e4 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:18.759847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.854865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa619238-6ca3-4a62-8f90-8bf9dbf3a110 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.878643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b0da-5991-4f42-b849-2e84bf76585d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:18.889491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.890165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.890453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.890739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.891479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:18.891791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:18.893232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:18.906171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 532/1070] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:18.911733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.924191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:18.925101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:18.925386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:18.925671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:18.926860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] RBAC: Authorizing `identity:get_project(project_id=b3401968aa89436194cb7f345df1d38c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:18.928635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:18.930290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 539/1071] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/projects/b3401968aa89436194cb7f345df1d38c => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:18.937125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:18.947024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:18.973401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:19.022723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68462f4-bbed-48c4-9f0f-5abb1fd88c31 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.044396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108548-1f4b-44a6-8730-e8101dd4817a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.064583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e227b3-d42e-4b48-9fbf-60e940357acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.065465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e227b3-d42e-4b48-9fbf-60e940357acf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.065747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e227b3-d42e-4b48-9fbf-60e940357acf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.066020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e227b3-d42e-4b48-9fbf-60e940357acf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.074594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e227b3-d42e-4b48-9fbf-60e940357acf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.076068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e227b3-d42e-4b48-9fbf-60e940357acf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.082065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934339dd-e7b4-4c5d-99ed-71105a324dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.083030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934339dd-e7b4-4c5d-99ed-71105a324dd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.083311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934339dd-e7b4-4c5d-99ed-71105a324dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.083582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934339dd-e7b4-4c5d-99ed-71105a324dd5 None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c/users/c186f06e18a14abdaa6c15d9eec3f1e4/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.085560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934339dd-e7b4-4c5d-99ed-71105a324dd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3401968aa89436194cb7f345df1d38c, user_id=c186f06e18a14abdaa6c15d9eec3f1e4, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.087043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934339dd-e7b4-4c5d-99ed-71105a324dd5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a8e227b3-d42e-4b48-9fbf-60e940357acf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:19.091369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:19.092634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 533/1072] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:26:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:19.093897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fa22c-080c-40b8-8d0e-8a7087b5cd4f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.094721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fa22c-080c-40b8-8d0e-8a7087b5cd4f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.094721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fa22c-080c-40b8-8d0e-8a7087b5cd4f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.095581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 534/1073] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:19.100544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fa22c-080c-40b8-8d0e-8a7087b5cd4f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.192290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:19.196678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 540/1074] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:26:18 2022] PUT /identity/v3/projects/b3401968aa89436194cb7f345df1d38c/users/c186f06e18a14abdaa6c15d9eec3f1e4/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:26:19.200631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5a04f-79ed-44ca-bd8d-8a7ce2ed8687 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.200631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5a04f-79ed-44ca-bd8d-8a7ce2ed8687 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.200631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5a04f-79ed-44ca-bd8d-8a7ce2ed8687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.205356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fa22c-080c-40b8-8d0e-8a7087b5cd4f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.232906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6fa5a04f-79ed-44ca-bd8d-8a7ce2ed8687 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:19.234147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6fa5a04f-79ed-44ca-bd8d-8a7ce2ed8687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:19.243680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.246727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.246727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.246727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.247158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:19.247481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.249031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.273933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 535/1075] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:19.279564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.327104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 541/1076] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:19.350660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5a04f-79ed-44ca-bd8d-8a7ce2ed8687 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.381153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:19.407020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25e454-4042-456b-b10a-1c1637157260 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.444555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.445298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.445578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.445853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.457645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.459411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.461583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5a04f-79ed-44ca-bd8d-8a7ce2ed8687 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.475862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:19.479890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:19.481297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:19.482261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:19.482261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:19.482261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:19.482261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:19.482261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 536/1077] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:19.487105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.501863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.502525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.502811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.503217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.577423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.578888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.596480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359243-d6be-40a1-aa83-187b3193653e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.603967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:19.633389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.634215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.634604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.634968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.635827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:19.636203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.637794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.655676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 537/1078] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:19.657567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.700575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 542/1079] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:26:19.706459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.733015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:19.775805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c67ac9-07b4-4e5e-aa57-82afbcab9316 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.815946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.817181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.817831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.818213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.828022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5aebf3-e79a-4a31-927a-cdf9df25b8d2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:19.828593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.830391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:19.850804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:19.851915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:19.852988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:19.853910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:19.854515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 538/1080] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:19.860035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.866103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:19.870599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:19.870599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:19.870599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:19.879148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:19.880815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:19.900467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:19.901583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:19.902480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:19.902480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:19.902480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:19.902480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:19.902480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 543/1081] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:19.908362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:19.979784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b9f11-2038-41b4-8f6d-2751e97a82f2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.004476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:20.019335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.019992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.020274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.020690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.021464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:20.021771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:20.023346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:20.023763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f261d-e4b5-486c-a2a9-68f812fe0162 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.040749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 539/1082] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:20.042533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:20.069952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e363d-f74b-4dd9-a087-e94ea61e4da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.070730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e363d-f74b-4dd9-a087-e94ea61e4da0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.071020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e363d-f74b-4dd9-a087-e94ea61e4da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.071297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e363d-f74b-4dd9-a087-e94ea61e4da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.072025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e363d-f74b-4dd9-a087-e94ea61e4da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:20.072342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e363d-f74b-4dd9-a087-e94ea61e4da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:20.074129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e363d-f74b-4dd9-a087-e94ea61e4da0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:20.087082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 544/1083] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:20.157661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0aed8-39e6-45de-9108-da5967104ce9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.195462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3f199-2ec1-474a-b80a-c2793fc63d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.196373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3f199-2ec1-474a-b80a-c2793fc63d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.196651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3f199-2ec1-474a-b80a-c2793fc63d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.196957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3f199-2ec1-474a-b80a-c2793fc63d08 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/ec899ae414744dde9ec9b76aac1ce9d8/roles/5c9ff5726a4f4a3c86b6504b08ec2239` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.201304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3f199-2ec1-474a-b80a-c2793fc63d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=ec899ae414744dde9ec9b76aac1ce9d8, role_id=5c9ff5726a4f4a3c86b6504b08ec2239)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:20.203937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3f199-2ec1-474a-b80a-c2793fc63d08 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:20.278301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 540/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:26:20 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/ec899ae414744dde9ec9b76aac1ce9d8/roles/5c9ff5726a4f4a3c86b6504b08ec2239 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:26:21.288718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a7b48-7322-469f-82a7-be8cae50bf91 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.289309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a7b48-7322-469f-82a7-be8cae50bf91 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.289309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a7b48-7322-469f-82a7-be8cae50bf91 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.290665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 545/1085] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:21.296616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d221cde-3d84-47dd-912c-c28cc15c4455 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.297050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d221cde-3d84-47dd-912c-c28cc15c4455 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.297205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d221cde-3d84-47dd-912c-c28cc15c4455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.313970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d221cde-3d84-47dd-912c-c28cc15c4455 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:21.315185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d221cde-3d84-47dd-912c-c28cc15c4455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:21.350564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.384113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 541/1086] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:21.407382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a7b48-7322-469f-82a7-be8cae50bf91 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.447274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.515181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a7b48-7322-469f-82a7-be8cae50bf91 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.551991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.552882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.553228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.553555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.566032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:21.575389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:21.576338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:21.577216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 546/1087] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:21.582738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d221cde-3d84-47dd-912c-c28cc15c4455 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.682200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d221cde-3d84-47dd-912c-c28cc15c4455 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.716886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.717552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.717835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.718110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.718863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:21.719166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:21.720545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.728619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 542/1088] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:21.734949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.834806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.841104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63263e-9dd7-4ca4-b2e5-96a9c3698073 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.877271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.878146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.878716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.878996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] PATH_INFO: `/v3/users/c186f06e18a14abdaa6c15d9eec3f1e4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.880194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] RBAC: Authorizing `identity:get_user(user_id=c186f06e18a14abdaa6c15d9eec3f1e4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:21.881718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.883371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 547/1089] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/users/c186f06e18a14abdaa6c15d9eec3f1e4 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:21.889269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.921840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.998724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1899ad-a286-412f-9269-5deccb88f494 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.033243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdc008-84c4-4654-98f1-a11a47ba060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:22.033895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc008-84c4-4654-98f1-a11a47ba060e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.034182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc008-84c4-4654-98f1-a11a47ba060e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.034462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc008-84c4-4654-98f1-a11a47ba060e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.107105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdc008-84c4-4654-98f1-a11a47ba060e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:22.108515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdc008-84c4-4654-98f1-a11a47ba060e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:22.146349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:22.216508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 543/1090] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:22.222450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:22.323665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7079bcf-2e97-4319-b05a-001dd1905dc0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.341679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537422c6-df85-4546-9774-c57866b085b7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.342211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537422c6-df85-4546-9774-c57866b085b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.342357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537422c6-df85-4546-9774-c57866b085b7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.343750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 544/1091] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:22.348636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7445d5e-d0b1-4e23-be53-995750255463 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.348953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7445d5e-d0b1-4e23-be53-995750255463 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.349259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7445d5e-d0b1-4e23-be53-995750255463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.362906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:22.363699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.363989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.364269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.365488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] RBAC: Authorizing `identity:get_project(project_id=b3401968aa89436194cb7f345df1d38c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:22.366024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f7445d5e-d0b1-4e23-be53-995750255463 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:22.366938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:22.367196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f7445d5e-d0b1-4e23-be53-995750255463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:22.368643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 548/1092] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:22 2022] GET /identity/v3/projects/b3401968aa89436194cb7f345df1d38c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:22.375654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:22.406154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:22.436314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 545/1093] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:22.459438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7445d5e-d0b1-4e23-be53-995750255463 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:22.476679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345898b8-b2fb-4d7a-afcd-d08425e73158 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.513310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:22.513975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.514256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.514539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.515362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=b3401968aa89436194cb7f345df1d38c, user.id=c186f06e18a14abdaa6c15d9eec3f1e4) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:22.515675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:22.517154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:22.523232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 549/1094] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:22 2022] GET /identity/v3/role_assignments?user.id=c186f06e18a14abdaa6c15d9eec3f1e4&scope.project.id=b3401968aa89436194cb7f345df1d38c&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:22.525035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:22.569106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7445d5e-d0b1-4e23-be53-995750255463 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.609615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:22.610451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.610735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.611010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.624759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:22.633109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:22.634270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:22.641054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 546/1095] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 18 10:26:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:22.642660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:22.803151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:22.808064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68c7b-5adb-4e7d-91c7-d1d683898f47 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.842347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:22.843014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.843298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.843573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.844323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:22.844629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:22.846038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:22.855676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 550/1096] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 18 10:26:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:22.861660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:22.968668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e28629-f9ad-4dc2-9daa-131dcf9b97f4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.991984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:23.003949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:23.004744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:23.005740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:23.005740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:23.017617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:23.019111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:23.037716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:23.038661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:23.039576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:23.039576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:23.039576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:23.039576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:23.041063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 547/1097] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:23.044935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:23.149993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8c71c-3267-4736-8b4d-10877d570ab9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:23.186038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:23.186693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:23.186974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:23.187246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:23.187969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:23.188270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:23.189664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:23.202174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 551/1098] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:23.207970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:23.310440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3020f6-edc3-42c3-aadf-3565f1cddf68 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:23.343546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:23.344333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:23.344614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:23.344913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:23.352317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:23.354303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:23.368068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:23.369296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:23.371255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 548/1099] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 18 10:26:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:23.376108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:23.394848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:23.474774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258d2d-de40-4bf8-a9bc-d19ceaaf86dc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:23.508842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:23.509508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:23.509791 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:23.510116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:23.511018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:23.511330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:23.512756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:23.530591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 552/1100] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 18 10:26:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:23.536477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:23.571932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:23.640653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7092c-03d6-4d15-bf14-91fd00cc23e0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:23.675588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:23.676252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:23.676533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:23.676834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:23.748605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:23.750103 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:23.783469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:23.850851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 549/1101] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:23.856645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:23.935980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:23.961363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8c04-f4aa-4305-b80d-af8497f1e684 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:23.996592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:23.997506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:23.998669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:23.998669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.009603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.011899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:24.030444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:24.031448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:24.032356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:24.032356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:24.032356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:24.032356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:24.032356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 553/1102] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:24.039503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.140627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dba73-d34f-4f05-b4b6-7dc84b43edf1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.175563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.176295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.176500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.176789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.177564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:24.177869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.179244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.180681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.191431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 550/1103] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:24.202279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.301407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47920cdd-9d96-4791-a946-9b81cc493f90 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.332456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.341261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.342163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.352460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.352460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.352460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.352460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:24.367458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:24.368454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:24.369952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 554/1104] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:24.375646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.476186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0c01a-966c-4005-b2af-86ad257f70c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.512127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43171426-ddb2-491d-aa6c-65bb18eb27a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.512802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43171426-ddb2-491d-aa6c-65bb18eb27a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.513099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43171426-ddb2-491d-aa6c-65bb18eb27a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.513375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43171426-ddb2-491d-aa6c-65bb18eb27a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.514160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43171426-ddb2-491d-aa6c-65bb18eb27a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:24.514496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43171426-ddb2-491d-aa6c-65bb18eb27a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.516027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43171426-ddb2-491d-aa6c-65bb18eb27a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.528466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 551/1105] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:24.535568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.598259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bdfce-380d-4e84-8686-fa353160d4b4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.598550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bdfce-380d-4e84-8686-fa353160d4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.598839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bdfce-380d-4e84-8686-fa353160d4b4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.600344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 552/1106] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:26:24.606093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996ece4-2288-4b9d-a94d-8cf60f4e3372 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.606379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996ece4-2288-4b9d-a94d-8cf60f4e3372 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.606656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996ece4-2288-4b9d-a94d-8cf60f4e3372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.627367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5996ece4-2288-4b9d-a94d-8cf60f4e3372 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:24.628572 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5996ece4-2288-4b9d-a94d-8cf60f4e3372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:24.645596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae89e7-b1f6-40f2-a366-f80d918b4a7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.681553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.683237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.683879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.684188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.684500 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.685374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] RBAC: Adding query filter params (role.id=5c9ff5726a4f4a3c86b6504b08ec2239, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=ec899ae414744dde9ec9b76aac1ce9d8) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:24.685690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.687143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.693541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 555/1107] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/role_assignments?user.id=ec899ae414744dde9ec9b76aac1ce9d8&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=5c9ff5726a4f4a3c86b6504b08ec2239 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:24.702370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 553/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:24.733671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.796014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.856285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c37f1-904f-4960-9fa8-50b98bdc8588 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.899901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e1372-4d79-4714-9399-54e18ed512f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.901711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1372-4d79-4714-9399-54e18ed512f8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.901995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1372-4d79-4714-9399-54e18ed512f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.902272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1372-4d79-4714-9399-54e18ed512f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.903521 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e1372-4d79-4714-9399-54e18ed512f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.905003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e1372-4d79-4714-9399-54e18ed512f8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.906902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 556/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:24.912967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996ece4-2288-4b9d-a94d-8cf60f4e3372 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:25.031076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996ece4-2288-4b9d-a94d-8cf60f4e3372 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:25.073369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4244c-bde7-4885-bc92-3dd85a72dddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:25.074965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4244c-bde7-4885-bc92-3dd85a72dddd None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:25.074965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4244c-bde7-4885-bc92-3dd85a72dddd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:25.074965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4244c-bde7-4885-bc92-3dd85a72dddd None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:25.076117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4244c-bde7-4885-bc92-3dd85a72dddd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:25.078142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4244c-bde7-4885-bc92-3dd85a72dddd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:25.198356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 554/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:26:24 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:27.413987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9e1c6-b00f-4774-aec5-0018d5d1c0b8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:27.414550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9e1c6-b00f-4774-aec5-0018d5d1c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:27.414762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9e1c6-b00f-4774-aec5-0018d5d1c0b8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:27.416361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 557/1111] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:27.421719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc91b8-91a1-4950-a745-ed737309e0c2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:27.422126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc91b8-91a1-4950-a745-ed737309e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:27.422279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc91b8-91a1-4950-a745-ed737309e0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:27.439337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-90bc91b8-91a1-4950-a745-ed737309e0c2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:27.440516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90bc91b8-91a1-4950-a745-ed737309e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:27.474356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:27.506488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 555/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:27.530629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9e1c6-b00f-4774-aec5-0018d5d1c0b8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:27.631368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9e1c6-b00f-4774-aec5-0018d5d1c0b8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:27.651133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:27.664386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:27.665248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:27.665530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:27.665816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] PATH_INFO: `/v3/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:27.668091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] RBAC: Authorizing `identity:get_role(role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:27.669466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:27.670827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 558/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:27 2022] GET /identity/v3/roles/4f582d9851794ff894a232e045525ec4 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:27.676854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc91b8-91a1-4950-a745-ed737309e0c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:27.775807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc91b8-91a1-4950-a745-ed737309e0c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:27.810957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:27.811879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:27.812158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:27.812431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] PATH_INFO: `/v3/users/e6be6751f80a4f5e80d5b7d7112f6c54` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:27.826152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] RBAC: Authorizing `identity:get_user(user_id=e6be6751f80a4f5e80d5b7d7112f6c54)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:27.827611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:27.829357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 556/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:27 2022] GET /identity/v3/users/e6be6751f80a4f5e80d5b7d7112f6c54 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:27.835227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:27.938953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9e36-26d8-457a-a8ea-615b818e7696 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:27.973583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:27.974227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:27.974508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:27.974780 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.139371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:28.152217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:28.153714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:28.252951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 559/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:28.258851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:28.287734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:28.359153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620da85e-fe93-4156-ae3b-6b10de39b656 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:28.394065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193de418-ed65-40b5-a356-95ab9518dcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:28.394925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193de418-ed65-40b5-a356-95ab9518dcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:28.395278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193de418-ed65-40b5-a356-95ab9518dcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:28.395558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193de418-ed65-40b5-a356-95ab9518dcc0 None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.396746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193de418-ed65-40b5-a356-95ab9518dcc0 None admin] RBAC: Authorizing `identity:get_project(project_id=b3401968aa89436194cb7f345df1d38c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:28.398207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193de418-ed65-40b5-a356-95ab9518dcc0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:28.407685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 557/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:28 2022] GET /identity/v3/projects/b3401968aa89436194cb7f345df1d38c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:28.407801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:28.493434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:28.506077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dea16-f5bc-44ae-be37-6cdcb6e1367c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:28.540076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b34323-78d4-4980-8001-cfdee59c2c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:28.540715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b34323-78d4-4980-8001-cfdee59c2c83 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:28.541701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b34323-78d4-4980-8001-cfdee59c2c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:28.542160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b34323-78d4-4980-8001-cfdee59c2c83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.542981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b34323-78d4-4980-8001-cfdee59c2c83 None admin] RBAC: Adding query filter params (role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=b3401968aa89436194cb7f345df1d38c, user.id=e6be6751f80a4f5e80d5b7d7112f6c54) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:28.543291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b34323-78d4-4980-8001-cfdee59c2c83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:28.544710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b34323-78d4-4980-8001-cfdee59c2c83 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:28.550053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 560/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:28 2022] GET /identity/v3/role_assignments?user.id=e6be6751f80a4f5e80d5b7d7112f6c54&scope.project.id=b3401968aa89436194cb7f345df1d38c&role.id=4f582d9851794ff894a232e045525ec4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:30.562132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb643d11-86a7-4df7-835d-1a18c2c59556 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:30.562658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb643d11-86a7-4df7-835d-1a18c2c59556 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:30.562658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb643d11-86a7-4df7-835d-1a18c2c59556 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:30.564076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 558/1118] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:30.569393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d73bd3-7e2f-4558-96df-5fb39bbb531b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:30.570002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d73bd3-7e2f-4558-96df-5fb39bbb531b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:30.570002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d73bd3-7e2f-4558-96df-5fb39bbb531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:30.586448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a0d73bd3-7e2f-4558-96df-5fb39bbb531b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:30.588207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0d73bd3-7e2f-4558-96df-5fb39bbb531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:30.654044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 561/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:30.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb643d11-86a7-4df7-835d-1a18c2c59556 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:30.771938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:30.779503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb643d11-86a7-4df7-835d-1a18c2c59556 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:30.815443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:30.816212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:30.816491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:30.816761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] PATH_INFO: `/v3/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:30.818240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] RBAC: Authorizing `identity:get_role(role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:30.819651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:30.821054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 559/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:30 2022] GET /identity/v3/roles/4f582d9851794ff894a232e045525ec4 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:30.827148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d73bd3-7e2f-4558-96df-5fb39bbb531b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:30.907426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:30.928245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d73bd3-7e2f-4558-96df-5fb39bbb531b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:30.962287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:30.963718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:30.963999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:30.964276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] PATH_INFO: `/v3/users/e6be6751f80a4f5e80d5b7d7112f6c54` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:30.965490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] RBAC: Authorizing `identity:get_user(user_id=e6be6751f80a4f5e80d5b7d7112f6c54)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:30.966926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:30.968886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 562/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:30 2022] GET /identity/v3/users/e6be6751f80a4f5e80d5b7d7112f6c54 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:30.974692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:31.075426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17318c97-8b54-4490-81ec-e367f19e4d3b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:31.109678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2febac8-3b80-4a14-a093-b14c2150981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:31.110519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2febac8-3b80-4a14-a093-b14c2150981c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:31.110796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2febac8-3b80-4a14-a093-b14c2150981c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:31.111071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2febac8-3b80-4a14-a093-b14c2150981c None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:31.112257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2febac8-3b80-4a14-a093-b14c2150981c None admin] RBAC: Authorizing `identity:get_project(project_id=b3401968aa89436194cb7f345df1d38c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:31.113736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2febac8-3b80-4a14-a093-b14c2150981c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:31.115404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 560/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:30 2022] GET /identity/v3/projects/b3401968aa89436194cb7f345df1d38c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:31.121965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:31.226006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc12a0a-f0b1-4cdb-b594-71c57037bdd4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:31.262683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29928565-ec3d-4fab-8839-2f04deab3d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:31.263571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29928565-ec3d-4fab-8839-2f04deab3d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:31.263859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29928565-ec3d-4fab-8839-2f04deab3d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:31.264142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29928565-ec3d-4fab-8839-2f04deab3d08 None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c/users/e6be6751f80a4f5e80d5b7d7112f6c54/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:31.266381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29928565-ec3d-4fab-8839-2f04deab3d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3401968aa89436194cb7f345df1d38c, user_id=e6be6751f80a4f5e80d5b7d7112f6c54, role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:31.267976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29928565-ec3d-4fab-8839-2f04deab3d08 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:31.348673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 563/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:26:31 2022] PUT /identity/v3/projects/b3401968aa89436194cb7f345df1d38c/users/e6be6751f80a4f5e80d5b7d7112f6c54/roles/4f582d9851794ff894a232e045525ec4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:26:33.370688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc638c04-f64a-4e6d-939a-b5aa3e9568c4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:33.371218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc638c04-f64a-4e6d-939a-b5aa3e9568c4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:33.371218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc638c04-f64a-4e6d-939a-b5aa3e9568c4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:33.372589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 561/1124] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:33.377913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092424-1ff3-432d-bb5c-b70597b7756a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:33.378329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092424-1ff3-432d-bb5c-b70597b7756a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:33.378472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092424-1ff3-432d-bb5c-b70597b7756a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:33.396010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cc092424-1ff3-432d-bb5c-b70597b7756a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:33.397245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc092424-1ff3-432d-bb5c-b70597b7756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:33.400207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:33.463872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 564/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:33.486589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc638c04-f64a-4e6d-939a-b5aa3e9568c4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:33.518891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:33.590788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc638c04-f64a-4e6d-939a-b5aa3e9568c4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:33.627039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:33.627809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:33.628090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:33.628467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] PATH_INFO: `/v3/roles/4f582d9851794ff894a232e045525ec4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:33.630136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] RBAC: Authorizing `identity:get_role(role_id=4f582d9851794ff894a232e045525ec4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:33.631522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:33.632916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 562/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:33 2022] GET /identity/v3/roles/4f582d9851794ff894a232e045525ec4 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:33.638893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc092424-1ff3-432d-bb5c-b70597b7756a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:33.739721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc092424-1ff3-432d-bb5c-b70597b7756a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:33.774156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:33.775094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:33.775376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:33.775649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] PATH_INFO: `/v3/users/e6be6751f80a4f5e80d5b7d7112f6c54` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:33.776877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] RBAC: Authorizing `identity:get_user(user_id=e6be6751f80a4f5e80d5b7d7112f6c54)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:33.778365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:33.779996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 565/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:33 2022] GET /identity/v3/users/e6be6751f80a4f5e80d5b7d7112f6c54 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:33.786284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:33.901441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:33.904338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad01e-2827-4545-ac38-4ab1d236d4ee None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:33.940330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:33.941020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:33.941310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:33.941575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:34.011591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:34.013081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:34.110760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:34.114576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 563/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:26:34.121953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:34.133006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:34.223287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345650f-64c2-46a7-b21f-b3c1c8d3d80f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:34.257996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467d0a8-7cf1-4b59-bbb3-8ec0803d3024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:34.258849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467d0a8-7cf1-4b59-bbb3-8ec0803d3024 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:34.259134 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467d0a8-7cf1-4b59-bbb3-8ec0803d3024 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:34.260036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467d0a8-7cf1-4b59-bbb3-8ec0803d3024 None admin] PATH_INFO: `/v3/projects/b3401968aa89436194cb7f345df1d38c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:34.261352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467d0a8-7cf1-4b59-bbb3-8ec0803d3024 None admin] RBAC: Authorizing `identity:get_project(project_id=b3401968aa89436194cb7f345df1d38c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:34.262793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467d0a8-7cf1-4b59-bbb3-8ec0803d3024 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:34.264498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 566/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:34 2022] GET /identity/v3/projects/b3401968aa89436194cb7f345df1d38c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:34.270814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:34.370804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a94e3-0885-47da-9644-dcc32b317c45 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:34.405782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07906dd7-b2d3-478b-9f49-7df7ce6a0b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:34.406428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07906dd7-b2d3-478b-9f49-7df7ce6a0b4f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:34.406713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07906dd7-b2d3-478b-9f49-7df7ce6a0b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:34.406986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07906dd7-b2d3-478b-9f49-7df7ce6a0b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:34.407813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07906dd7-b2d3-478b-9f49-7df7ce6a0b4f None admin] RBAC: Adding query filter params (role.id=4f582d9851794ff894a232e045525ec4, scope.project.id=b3401968aa89436194cb7f345df1d38c, user.id=e6be6751f80a4f5e80d5b7d7112f6c54) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:34.408127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07906dd7-b2d3-478b-9f49-7df7ce6a0b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:34.409707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07906dd7-b2d3-478b-9f49-7df7ce6a0b4f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:34.415611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 564/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:34 2022] GET /identity/v3/role_assignments?user.id=e6be6751f80a4f5e80d5b7d7112f6c54&scope.project.id=b3401968aa89436194cb7f345df1d38c&role.id=4f582d9851794ff894a232e045525ec4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:36.447883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae475-61e3-4157-9d66-c431581a3a5d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:36.448406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae475-61e3-4157-9d66-c431581a3a5d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:36.448406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae475-61e3-4157-9d66-c431581a3a5d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:36.449964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 567/1131] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:36.455284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259105f-1f3a-4155-891f-7265b2ce1657 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:36.455713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259105f-1f3a-4155-891f-7265b2ce1657 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:36.455847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259105f-1f3a-4155-891f-7265b2ce1657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:36.472502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d259105f-1f3a-4155-891f-7265b2ce1657 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:36.473724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d259105f-1f3a-4155-891f-7265b2ce1657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:36.531342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:36.539411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 565/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:36.563809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aae475-61e3-4157-9d66-c431581a3a5d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:36.641891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:36.664686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aae475-61e3-4157-9d66-c431581a3a5d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:36.698092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deeecc-1775-45b1-8af6-84f6b4ff46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:36.698816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deeecc-1775-45b1-8af6-84f6b4ff46de None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:36.699099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deeecc-1775-45b1-8af6-84f6b4ff46de None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:36.699375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deeecc-1775-45b1-8af6-84f6b4ff46de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:36.700562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21deeecc-1775-45b1-8af6-84f6b4ff46de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:36.702035 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21deeecc-1775-45b1-8af6-84f6b4ff46de None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:36.704063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 568/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:36.709877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259105f-1f3a-4155-891f-7265b2ce1657 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:36.812058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259105f-1f3a-4155-891f-7265b2ce1657 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:36.846255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c3894-1ac5-4082-b261-66ea66f85cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:36.847053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c3894-1ac5-4082-b261-66ea66f85cef None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:36.847437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c3894-1ac5-4082-b261-66ea66f85cef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:36.847819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c3894-1ac5-4082-b261-66ea66f85cef None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:36.848657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c3894-1ac5-4082-b261-66ea66f85cef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:36.850314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c3894-1ac5-4082-b261-66ea66f85cef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:36.981062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 566/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:26:36 2022] POST /identity/v3/projects => generated 311 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:39.093092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c9b2-cb44-4c75-a4b3-9de76e740eb1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:39.093596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c9b2-cb44-4c75-a4b3-9de76e740eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:39.093596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c9b2-cb44-4c75-a4b3-9de76e740eb1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:39.094947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 569/1135] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:39.100216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d28c3d-5dc8-4e12-a986-66cff59cbc80 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:39.100634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d28c3d-5dc8-4e12-a986-66cff59cbc80 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:39.100770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d28c3d-5dc8-4e12-a986-66cff59cbc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:39.118122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-14d28c3d-5dc8-4e12-a986-66cff59cbc80 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:39.120401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-14d28c3d-5dc8-4e12-a986-66cff59cbc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:39.161759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:39.186904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 567/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:39.212574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73c9b2-cb44-4c75-a4b3-9de76e740eb1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:39.318220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73c9b2-cb44-4c75-a4b3-9de76e740eb1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:39.353751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9850b-fa88-45e8-9dc5-4a7109d8c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:39.354462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9850b-fa88-45e8-9dc5-4a7109d8c133 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:39.354744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9850b-fa88-45e8-9dc5-4a7109d8c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:39.355016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9850b-fa88-45e8-9dc5-4a7109d8c133 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:39.356204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9850b-fa88-45e8-9dc5-4a7109d8c133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:39.357678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9850b-fa88-45e8-9dc5-4a7109d8c133 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:39.359293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 570/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:39.365324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d28c3d-5dc8-4e12-a986-66cff59cbc80 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:39.464492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d28c3d-5dc8-4e12-a986-66cff59cbc80 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:39.499161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256480f-87ab-4935-8616-bd6b8e787281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:39.499837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256480f-87ab-4935-8616-bd6b8e787281 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:39.500126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256480f-87ab-4935-8616-bd6b8e787281 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:39.500404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256480f-87ab-4935-8616-bd6b8e787281 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:39.501161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256480f-87ab-4935-8616-bd6b8e787281 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:39.502687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256480f-87ab-4935-8616-bd6b8e787281 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:39.597835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 568/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:26:39 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:41.731345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4e6de-458f-4e8b-bfbd-d45afa0ddd9c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:41.731879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4e6de-458f-4e8b-bfbd-d45afa0ddd9c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:41.731879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4e6de-458f-4e8b-bfbd-d45afa0ddd9c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:41.733301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 571/1139] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:41.738802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b1f9-a25d-4ed0-94d7-87e79d58d9f6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:41.739201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b1f9-a25d-4ed0-94d7-87e79d58d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:41.739364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b1f9-a25d-4ed0-94d7-87e79d58d9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:41.756554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-43a5b1f9-a25d-4ed0-94d7-87e79d58d9f6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:41.757768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43a5b1f9-a25d-4ed0-94d7-87e79d58d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:41.785926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:41.828561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 569/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:41.853172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4e6de-458f-4e8b-bfbd-d45afa0ddd9c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:41.868414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:41.959841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4e6de-458f-4e8b-bfbd-d45afa0ddd9c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:41.997022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:41.997814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:41.998110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:41.998390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:42.009936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:42.018010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:42.019433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:42.021014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 572/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:42.026930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5b1f9-a25d-4ed0-94d7-87e79d58d9f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:42.129462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5b1f9-a25d-4ed0-94d7-87e79d58d9f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:42.166874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:42.167540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:42.167822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:42.168094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:42.168869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:42.169177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:42.170596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:42.189641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 570/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:42.195735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:42.268198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:42.299122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecbd2a-198f-4d21-8bb0-142349afc9d2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:42.335232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:42.336104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:42.336386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:42.336664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] PATH_INFO: `/v3/users/241e74c363d5467ea77866bea17f6bf1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:42.351675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] RBAC: Authorizing `identity:get_user(user_id=241e74c363d5467ea77866bea17f6bf1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:42.354003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:42.355768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 573/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:42 2022] GET /identity/v3/users/241e74c363d5467ea77866bea17f6bf1 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:42.361887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:42.387604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:42.466222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebffc9f-f208-44e6-9f9b-c3c02318389d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:42.501004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:42.501658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:42.501997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:42.502281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:42.571919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:42.573333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:42.597303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:42.675047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 571/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:26:42.681822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:42.786357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89adf76-59f6-4162-89e8-9f7c26062be6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:42.818630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:42.827700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b0ab8-162b-4c4b-8f2a-300313d0b785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:42.827700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b0ab8-162b-4c4b-8f2a-300313d0b785 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:42.827700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b0ab8-162b-4c4b-8f2a-300313d0b785 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:42.827700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b0ab8-162b-4c4b-8f2a-300313d0b785 None admin] PATH_INFO: `/v3/projects/d83554039c4348699abac8a540d4935c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:42.828104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b0ab8-162b-4c4b-8f2a-300313d0b785 None admin] RBAC: Authorizing `identity:get_project(project_id=d83554039c4348699abac8a540d4935c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:42.829070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b0ab8-162b-4c4b-8f2a-300313d0b785 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:42.830778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 574/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:42 2022] GET /identity/v3/projects/d83554039c4348699abac8a540d4935c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:42.837600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:42.943858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bb576-697b-4d19-9334-9a230163a63b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:42.968662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:42.979856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a9be7-01b2-4aec-b117-f82341226a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:42.987881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9be7-01b2-4aec-b117-f82341226a80 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:42.987881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9be7-01b2-4aec-b117-f82341226a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:42.987881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9be7-01b2-4aec-b117-f82341226a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:42.987881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9be7-01b2-4aec-b117-f82341226a80 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=d83554039c4348699abac8a540d4935c, user.id=241e74c363d5467ea77866bea17f6bf1) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:42.987881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9be7-01b2-4aec-b117-f82341226a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:42.987881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9be7-01b2-4aec-b117-f82341226a80 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:42.989025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 572/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:42 2022] GET /identity/v3/role_assignments?user.id=241e74c363d5467ea77866bea17f6bf1&scope.project.id=d83554039c4348699abac8a540d4935c&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:45.028532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ae9e1-2a5c-47a3-8c46-751e38d55508 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:45.029215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ae9e1-2a5c-47a3-8c46-751e38d55508 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:45.029215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ae9e1-2a5c-47a3-8c46-751e38d55508 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:45.030472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 575/1147] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:45.035738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc4ee2-69ea-49f8-8ea2-ca9ed716e6c7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:45.036244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc4ee2-69ea-49f8-8ea2-ca9ed716e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:45.036352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc4ee2-69ea-49f8-8ea2-ca9ed716e6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:45.052947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51bc4ee2-69ea-49f8-8ea2-ca9ed716e6c7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:45.054044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51bc4ee2-69ea-49f8-8ea2-ca9ed716e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:45.120399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 573/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:45.143368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ae9e1-2a5c-47a3-8c46-751e38d55508 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:45.245955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ae9e1-2a5c-47a3-8c46-751e38d55508 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:45.280025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:45.280837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:45.281125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:45.281399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:45.291440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:45.298921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:45.300014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:45.301015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 576/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:45.307689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc4ee2-69ea-49f8-8ea2-ca9ed716e6c7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:45.406613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc4ee2-69ea-49f8-8ea2-ca9ed716e6c7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:45.423994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:45.442750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:45.443417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:45.443700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:45.443973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:45.444828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:45.445143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:45.447262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:45.455280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 574/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:45.461271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:45.519500 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:45.561364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1353b-d266-458d-a31a-0932b3d4e705 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:45.593643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:45.594461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:45.594746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:45.595019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] PATH_INFO: `/v3/users/241e74c363d5467ea77866bea17f6bf1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:45.596212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] RBAC: Authorizing `identity:get_user(user_id=241e74c363d5467ea77866bea17f6bf1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:45.597739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:45.599388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 577/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:45 2022] GET /identity/v3/users/241e74c363d5467ea77866bea17f6bf1 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:45.606508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:45.705888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe37661-3843-466e-b92a-688bae584050 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:45.741849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cbc50-3285-41f6-ae3d-ced5c9d4ae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:45.742715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cbc50-3285-41f6-ae3d-ced5c9d4ae95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:45.743614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cbc50-3285-41f6-ae3d-ced5c9d4ae95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:45.743949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cbc50-3285-41f6-ae3d-ced5c9d4ae95 None admin] PATH_INFO: `/v3/projects/d83554039c4348699abac8a540d4935c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:45.745578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7cbc50-3285-41f6-ae3d-ced5c9d4ae95 None admin] RBAC: Authorizing `identity:get_project(project_id=d83554039c4348699abac8a540d4935c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:45.747032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7cbc50-3285-41f6-ae3d-ced5c9d4ae95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:45.748725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 575/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:45 2022] GET /identity/v3/projects/d83554039c4348699abac8a540d4935c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:45.755182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:45.857511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2cd0f-52db-4de1-b19a-5a7616bde56a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:45.883270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:45.892155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06127878-3060-4512-9e9b-a1dfc7441c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:45.899596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06127878-3060-4512-9e9b-a1dfc7441c2f None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:45.899596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06127878-3060-4512-9e9b-a1dfc7441c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:45.899596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06127878-3060-4512-9e9b-a1dfc7441c2f None admin] PATH_INFO: `/v3/projects/d83554039c4348699abac8a540d4935c/users/241e74c363d5467ea77866bea17f6bf1/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:45.899596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06127878-3060-4512-9e9b-a1dfc7441c2f None admin] RBAC: Authorizing `identity:create_grant(project_id=d83554039c4348699abac8a540d4935c, user_id=241e74c363d5467ea77866bea17f6bf1, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:45.899596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06127878-3060-4512-9e9b-a1dfc7441c2f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:46.004910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 578/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:26:45 2022] PUT /identity/v3/projects/d83554039c4348699abac8a540d4935c/users/241e74c363d5467ea77866bea17f6bf1/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:26:48.082732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269fafe-9885-404c-95c5-b8f126b47db7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:48.083232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269fafe-9885-404c-95c5-b8f126b47db7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:48.083232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269fafe-9885-404c-95c5-b8f126b47db7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:48.084673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 576/1154] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:48.090554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc25c1-546a-4070-b238-a4d99a18a882 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:48.091029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc25c1-546a-4070-b238-a4d99a18a882 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:48.091129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc25c1-546a-4070-b238-a4d99a18a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:48.108761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-51cc25c1-546a-4070-b238-a4d99a18a882 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:48.109973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51cc25c1-546a-4070-b238-a4d99a18a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:48.185501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 579/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:48.209545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269fafe-9885-404c-95c5-b8f126b47db7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:48.261594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:48.316741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269fafe-9885-404c-95c5-b8f126b47db7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:48.353670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:48.354542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:48.354916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:48.355273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:48.365473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:48.374318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:48.375816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:26:48.377230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 577/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:48.383671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc25c1-546a-4070-b238-a4d99a18a882 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:48.499546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc25c1-546a-4070-b238-a4d99a18a882 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:48.518729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:48.538662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:48.539339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:48.539680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:48.539993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:48.540981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:48.541360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:48.543120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:48.551920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 580/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:48.558243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:48.660288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a38aba-a485-4b38-86ca-ac50f1caa38b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:48.669355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:48.694073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:48.694937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:48.695214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:48.695486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] PATH_INFO: `/v3/users/241e74c363d5467ea77866bea17f6bf1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:48.697746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] RBAC: Authorizing `identity:get_user(user_id=241e74c363d5467ea77866bea17f6bf1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:48.699190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:48.700936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 578/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:26:48 2022] GET /identity/v3/users/241e74c363d5467ea77866bea17f6bf1 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:48.706907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:48.811315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498eca8-33d0-401a-a9bc-7d96057f0bb9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:48.849614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:48.850265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:48.850546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:48.850818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:48.898351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:48.920358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:48.921770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:49.024563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 581/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:26:49.032584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:49.130604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455410-6663-462b-8c3b-fa9985752115 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:49.164108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e85daa-bbd8-4bcc-a884-eb7d27beaaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:49.165430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85daa-bbd8-4bcc-a884-eb7d27beaaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:49.165713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85daa-bbd8-4bcc-a884-eb7d27beaaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:49.165988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85daa-bbd8-4bcc-a884-eb7d27beaaa5 None admin] PATH_INFO: `/v3/projects/d83554039c4348699abac8a540d4935c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:49.167167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e85daa-bbd8-4bcc-a884-eb7d27beaaa5 None admin] RBAC: Authorizing `identity:get_project(project_id=d83554039c4348699abac8a540d4935c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:49.168610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e85daa-bbd8-4bcc-a884-eb7d27beaaa5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:49.170418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 579/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:26:49 2022] GET /identity/v3/projects/d83554039c4348699abac8a540d4935c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:49.176680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:49.256285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:49.278813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6796a-3b68-4e5c-9fe3-d56776d408cf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:49.314352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944727ab-e762-486b-9c26-caf1115404ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:49.315003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944727ab-e762-486b-9c26-caf1115404ba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:49.315290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944727ab-e762-486b-9c26-caf1115404ba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:49.315569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944727ab-e762-486b-9c26-caf1115404ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:49.316397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944727ab-e762-486b-9c26-caf1115404ba None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=d83554039c4348699abac8a540d4935c, user.id=241e74c363d5467ea77866bea17f6bf1) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:49.316716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944727ab-e762-486b-9c26-caf1115404ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:49.318174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944727ab-e762-486b-9c26-caf1115404ba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:49.324052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 582/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:49 2022] GET /identity/v3/role_assignments?user.id=241e74c363d5467ea77866bea17f6bf1&scope.project.id=d83554039c4348699abac8a540d4935c&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:51.346022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349df5c-3ec5-4459-b536-7345d27b0f75 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.346656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349df5c-3ec5-4459-b536-7345d27b0f75 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.346656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349df5c-3ec5-4459-b536-7345d27b0f75 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.348010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 580/1162] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:51.354762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce982be-c68d-46f4-be28-d13f198317d4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.355211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce982be-c68d-46f4-be28-d13f198317d4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.355322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce982be-c68d-46f4-be28-d13f198317d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.373063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5ce982be-c68d-46f4-be28-d13f198317d4 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:51.374249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5ce982be-c68d-46f4-be28-d13f198317d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:51.441367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 583/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:51.463914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349df5c-3ec5-4459-b536-7345d27b0f75 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:51.511640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:51.570304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349df5c-3ec5-4459-b536-7345d27b0f75 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:51.606468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:51.607039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.607325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.607631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.675687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:51.677177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:51.721418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:51.776803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 581/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:26:51.782830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce982be-c68d-46f4-be28-d13f198317d4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:51.854891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:51.883917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce982be-c68d-46f4-be28-d13f198317d4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:51.920571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e667df8-8ece-410b-8496-1a628205c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:51.921420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e667df8-8ece-410b-8496-1a628205c234 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.921701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e667df8-8ece-410b-8496-1a628205c234 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.921977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e667df8-8ece-410b-8496-1a628205c234 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.929037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e667df8-8ece-410b-8496-1a628205c234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:51.931191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e667df8-8ece-410b-8496-1a628205c234 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7e667df8-8ece-410b-8496-1a628205c234 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:51.946436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:51.951880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:26:51.953080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 584/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:26:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:51.954158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:52.052440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dee4a-7de7-42ea-bd8d-5d7364e92974 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:52.087432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264e2ef-a3d6-400c-8dbc-1fa6c323b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:52.088124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264e2ef-a3d6-400c-8dbc-1fa6c323b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:52.088405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264e2ef-a3d6-400c-8dbc-1fa6c323b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:52.088679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264e2ef-a3d6-400c-8dbc-1fa6c323b1c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:52.089437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264e2ef-a3d6-400c-8dbc-1fa6c323b1c1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:52.089742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264e2ef-a3d6-400c-8dbc-1fa6c323b1c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:52.091116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264e2ef-a3d6-400c-8dbc-1fa6c323b1c1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:52.098283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 582/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 18 10:26:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:54.112339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e18c6-4d53-486a-ad4e-c389653c71ed None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:54.112889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e18c6-4d53-486a-ad4e-c389653c71ed None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:54.112889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e18c6-4d53-486a-ad4e-c389653c71ed None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:54.114576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 585/1167] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:54.120630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2b37-928d-44da-995b-82b50e1c62e6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:54.121074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2b37-928d-44da-995b-82b50e1c62e6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:54.121230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2b37-928d-44da-995b-82b50e1c62e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:54.138657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d49c2b37-928d-44da-995b-82b50e1c62e6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:54.139839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d49c2b37-928d-44da-995b-82b50e1c62e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:54.141843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:54.206846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 583/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:54.231424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e18c6-4d53-486a-ad4e-c389653c71ed None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:54.335509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e18c6-4d53-486a-ad4e-c389653c71ed None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:54.365877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:54.370970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe702ba-acf0-4fe7-91d0-f2ec272c9161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:54.371650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe702ba-acf0-4fe7-91d0-f2ec272c9161 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:54.371947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe702ba-acf0-4fe7-91d0-f2ec272c9161 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:54.372220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe702ba-acf0-4fe7-91d0-f2ec272c9161 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:54.373003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe702ba-acf0-4fe7-91d0-f2ec272c9161 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:54.374388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe702ba-acf0-4fe7-91d0-f2ec272c9161 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:54.515059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 586/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Mon Jul 18 10:26:54 2022] POST /identity/v3/domains => generated 267 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:56.562226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897e60d-7e6a-4501-a109-3e44226412d2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:56.562757 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897e60d-7e6a-4501-a109-3e44226412d2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:56.562757 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897e60d-7e6a-4501-a109-3e44226412d2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:56.564150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 584/1170] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:56.569621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22722f0d-6b70-40b5-b14d-bab659559160 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:56.570166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22722f0d-6b70-40b5-b14d-bab659559160 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:56.570166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22722f0d-6b70-40b5-b14d-bab659559160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:56.587347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22722f0d-6b70-40b5-b14d-bab659559160 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:56.588541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22722f0d-6b70-40b5-b14d-bab659559160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:56.655180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 587/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:56.679540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897e60d-7e6a-4501-a109-3e44226412d2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:56.781976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897e60d-7e6a-4501-a109-3e44226412d2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:56.816675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62e1ed-b677-4912-aa6c-a366653512c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:56.817499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e1ed-b677-4912-aa6c-a366653512c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:56.817778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e1ed-b677-4912-aa6c-a366653512c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:56.818051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e1ed-b677-4912-aa6c-a366653512c7 None admin] PATH_INFO: `/v3/domains/1f931d50c676499b940c775c13000982` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:56.825435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:56.829536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62e1ed-b677-4912-aa6c-a366653512c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f931d50c676499b940c775c13000982)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:56.838641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62e1ed-b677-4912-aa6c-a366653512c7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:56.838641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 585/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Jul 18 10:26:56 2022] GET /identity/v3/domains/1f931d50c676499b940c775c13000982 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:56.839772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22722f0d-6b70-40b5-b14d-bab659559160 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:56.939005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22722f0d-6b70-40b5-b14d-bab659559160 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:56.963664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:56.974799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfe34a-175f-445f-846a-da1eb7fca665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:56.982068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfe34a-175f-445f-846a-da1eb7fca665 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:56.982068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfe34a-175f-445f-846a-da1eb7fca665 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:56.982068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfe34a-175f-445f-846a-da1eb7fca665 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:56.982068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfe34a-175f-445f-846a-da1eb7fca665 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:56.982068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfe34a-175f-445f-846a-da1eb7fca665 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:57.062967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 588/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:26:56 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:59.141521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e734e3-68ab-4eac-ab44-449901caf388 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:59.142040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e734e3-68ab-4eac-ab44-449901caf388 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:59.142040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e734e3-68ab-4eac-ab44-449901caf388 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:59.143417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 586/1174] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:59.148829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335d3e5-875d-4440-917f-972360909895 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:59.149281 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335d3e5-875d-4440-917f-972360909895 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:59.149566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335d3e5-875d-4440-917f-972360909895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:59.166551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f335d3e5-875d-4440-917f-972360909895 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:59.167721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f335d3e5-875d-4440-917f-972360909895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:59.234239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 589/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:59.259471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e734e3-68ab-4eac-ab44-449901caf388 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:59.360950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e734e3-68ab-4eac-ab44-449901caf388 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:59.396005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6dae2-7f9c-4772-b03f-15ff0c550daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:59.396810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6dae2-7f9c-4772-b03f-15ff0c550daa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:59.397099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6dae2-7f9c-4772-b03f-15ff0c550daa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:59.397376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6dae2-7f9c-4772-b03f-15ff0c550daa None admin] PATH_INFO: `/v3/domains/1f931d50c676499b940c775c13000982` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:59.399079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6dae2-7f9c-4772-b03f-15ff0c550daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f931d50c676499b940c775c13000982)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:59.400526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6dae2-7f9c-4772-b03f-15ff0c550daa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:59.402171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 587/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Jul 18 10:26:59 2022] GET /identity/v3/domains/1f931d50c676499b940c775c13000982 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:59.408022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335d3e5-875d-4440-917f-972360909895 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:59.506834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335d3e5-875d-4440-917f-972360909895 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:59.509582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:59.542544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2622247-a14b-49f3-a81f-89032214f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:59.543300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2622247-a14b-49f3-a81f-89032214f9bf None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:59.543607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2622247-a14b-49f3-a81f-89032214f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:59.543901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2622247-a14b-49f3-a81f-89032214f9bf None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:59.544661 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2622247-a14b-49f3-a81f-89032214f9bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:59.546209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2622247-a14b-49f3-a81f-89032214f9bf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:59.720212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 590/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:26:59 2022] POST /identity/v3/users => generated 311 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:27:01.786388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95973768-9adf-4157-8722-a352fe2afa53 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:01.786947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95973768-9adf-4157-8722-a352fe2afa53 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:01.787060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95973768-9adf-4157-8722-a352fe2afa53 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:01.788365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 588/1178] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:01.794308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0261-2ced-43e5-be9a-2db12c919c23 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:01.794767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0261-2ced-43e5-be9a-2db12c919c23 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:01.794970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0261-2ced-43e5-be9a-2db12c919c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:01.813352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-15ce0261-2ced-43e5-be9a-2db12c919c23 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:01.814557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-15ce0261-2ced-43e5-be9a-2db12c919c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:01.881564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 591/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:27:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:27:01.905412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95973768-9adf-4157-8722-a352fe2afa53 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:01.973607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:02.010863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95973768-9adf-4157-8722-a352fe2afa53 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:02.046859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:02.047554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.047834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.048105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.058627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:27:02.066178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:27:02.067362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:27:02.068264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 589/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:27:02.074552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce0261-2ced-43e5-be9a-2db12c919c23 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:02.167339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:02.176541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce0261-2ced-43e5-be9a-2db12c919c23 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:02.210413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:02.211144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.211435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.211711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.212460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:27:02.212770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:02.214168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:02.222070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 592/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:02.228596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:02.328091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a7ed1-e30a-4027-871e-87df1893ea62 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:02.353272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:02.361893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:02.362765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.374230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.374230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] PATH_INFO: `/v3/users/0fe481e3829f459fa524dd05e60d261b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.377366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] RBAC: Authorizing `identity:get_user(user_id=0fe481e3829f459fa524dd05e60d261b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:02.378823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:02.380772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 590/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:27:02 2022] GET /identity/v3/users/0fe481e3829f459fa524dd05e60d261b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:02.386796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:02.488015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07838abc-8474-45c4-b2c8-161dda84bedc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:02.520648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:02.521319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.521603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.521877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.543633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:02.588831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:02.590730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:02.691201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 593/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:27:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:27:02.697473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:02.799395 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765744-0d84-4b01-ab6d-74be312a2217 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:02.834545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8ed54-60dd-414e-b311-87e4bd7eef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:02.835398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8ed54-60dd-414e-b311-87e4bd7eef6a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.836246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8ed54-60dd-414e-b311-87e4bd7eef6a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.836520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8ed54-60dd-414e-b311-87e4bd7eef6a None admin] PATH_INFO: `/v3/projects/b22934123a9447cc91ab604a5d2cea3c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.837730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8ed54-60dd-414e-b311-87e4bd7eef6a None admin] RBAC: Authorizing `identity:get_project(project_id=b22934123a9447cc91ab604a5d2cea3c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:02.839294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8ed54-60dd-414e-b311-87e4bd7eef6a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:02.840955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 591/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:27:02 2022] GET /identity/v3/projects/b22934123a9447cc91ab604a5d2cea3c => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:02.847185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:02.899453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:02.945860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2831f-1bba-4f0f-893c-746f37291a4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:02.980057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063ff4f-8c5e-47c5-a82a-d92ac8bb8724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:02.980710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ff4f-8c5e-47c5-a82a-d92ac8bb8724 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.981023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ff4f-8c5e-47c5-a82a-d92ac8bb8724 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.981303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ff4f-8c5e-47c5-a82a-d92ac8bb8724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.982133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063ff4f-8c5e-47c5-a82a-d92ac8bb8724 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=b22934123a9447cc91ab604a5d2cea3c, user.id=0fe481e3829f459fa524dd05e60d261b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:27:02.982445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063ff4f-8c5e-47c5-a82a-d92ac8bb8724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:02.983882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063ff4f-8c5e-47c5-a82a-d92ac8bb8724 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:02.990126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 594/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:27:02 2022] GET /identity/v3/role_assignments?user.id=0fe481e3829f459fa524dd05e60d261b&scope.project.id=b22934123a9447cc91ab604a5d2cea3c&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:05.014554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb03995-9928-4cd5-8f6a-025f023f9e89 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:05.015098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb03995-9928-4cd5-8f6a-025f023f9e89 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:05.015098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb03995-9928-4cd5-8f6a-025f023f9e89 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:05.016446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 592/1186] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:05.021559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36968f4-e1e7-45b8-aa17-35036afb7fb5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:05.021961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36968f4-e1e7-45b8-aa17-35036afb7fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:05.022119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36968f4-e1e7-45b8-aa17-35036afb7fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:05.039255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a36968f4-e1e7-45b8-aa17-35036afb7fb5 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:05.040421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a36968f4-e1e7-45b8-aa17-35036afb7fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:05.106646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 595/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:27:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:27:05.130211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb03995-9928-4cd5-8f6a-025f023f9e89 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:05.184564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:05.234844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb03995-9928-4cd5-8f6a-025f023f9e89 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:05.269316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:05.270096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:05.270377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:05.270652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:05.280975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:27:05.289197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:27:05.290440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:27:05.291301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 593/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:27:05.296532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36968f4-e1e7-45b8-aa17-35036afb7fb5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:05.343418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:05.398440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36968f4-e1e7-45b8-aa17-35036afb7fb5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:05.432930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:05.433589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:05.433873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:05.434146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:05.434890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:27:05.435194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:05.436592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:05.444645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 596/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:27:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:05.450627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:05.551353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaef34b-d97d-423f-931f-7b1a9866940f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:05.564054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:05.585861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:05.586889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:05.587170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:05.587444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] PATH_INFO: `/v3/users/0fe481e3829f459fa524dd05e60d261b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:05.588630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] RBAC: Authorizing `identity:get_user(user_id=0fe481e3829f459fa524dd05e60d261b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:05.590117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:05.591745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 594/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:27:05 2022] GET /identity/v3/users/0fe481e3829f459fa524dd05e60d261b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:05.597683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:05.699412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf68cf-3348-4805-9822-dfce6b158e8d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:05.708214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:05.734339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7092d4-36b0-4d87-aa9a-ac3f72b5f7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:05.735206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7092d4-36b0-4d87-aa9a-ac3f72b5f7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:05.735560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7092d4-36b0-4d87-aa9a-ac3f72b5f7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:05.735864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7092d4-36b0-4d87-aa9a-ac3f72b5f7f3 None admin] PATH_INFO: `/v3/projects/b22934123a9447cc91ab604a5d2cea3c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:05.737198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7092d4-36b0-4d87-aa9a-ac3f72b5f7f3 None admin] RBAC: Authorizing `identity:get_project(project_id=b22934123a9447cc91ab604a5d2cea3c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:05.738646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7092d4-36b0-4d87-aa9a-ac3f72b5f7f3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:05.740384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 597/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:27:05 2022] GET /identity/v3/projects/b22934123a9447cc91ab604a5d2cea3c => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:05.747831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:05.847431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b8da2-702c-4dca-8bc8-56d195664b92 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:05.882374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274bd69-b130-4b1c-84df-725a8fa8cb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:05.883334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274bd69-b130-4b1c-84df-725a8fa8cb4f None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:05.883615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274bd69-b130-4b1c-84df-725a8fa8cb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:05.883889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274bd69-b130-4b1c-84df-725a8fa8cb4f None admin] PATH_INFO: `/v3/projects/b22934123a9447cc91ab604a5d2cea3c/users/0fe481e3829f459fa524dd05e60d261b/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:05.885839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274bd69-b130-4b1c-84df-725a8fa8cb4f None admin] RBAC: Authorizing `identity:create_grant(project_id=b22934123a9447cc91ab604a5d2cea3c, user_id=0fe481e3829f459fa524dd05e60d261b, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:05.887291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274bd69-b130-4b1c-84df-725a8fa8cb4f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:05.993743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 595/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:27:05 2022] PUT /identity/v3/projects/b22934123a9447cc91ab604a5d2cea3c/users/0fe481e3829f459fa524dd05e60d261b/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:27:07.995656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4beed6-9131-4ecc-ad26-6614174bbe4f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:07.996246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4beed6-9131-4ecc-ad26-6614174bbe4f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:07.996246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4beed6-9131-4ecc-ad26-6614174bbe4f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:07.997659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 598/1193] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:08.002989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9b973-c0a9-403d-97f7-a0c2c05e0107 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:08.003446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9b973-c0a9-403d-97f7-a0c2c05e0107 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:08.003540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9b973-c0a9-403d-97f7-a0c2c05e0107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:08.020320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e8c9b973-c0a9-403d-97f7-a0c2c05e0107 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:08.021531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8c9b973-c0a9-403d-97f7-a0c2c05e0107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:08.056893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:08.089836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 596/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:27:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:27:08.114153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4beed6-9131-4ecc-ad26-6614174bbe4f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:08.216869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4beed6-9131-4ecc-ad26-6614174bbe4f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:08.250356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:08.251139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:08.251422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:08.251697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:08.262457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:27:08.270028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:27:08.271115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:27:08.272198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 599/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:27:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:27:08.277676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9b973-c0a9-403d-97f7-a0c2c05e0107 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:08.377143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9b973-c0a9-403d-97f7-a0c2c05e0107 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:08.411714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:08.412371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:08.412889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:08.413421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:08.414821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:27:08.415215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:08.416650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:08.424789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 597/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:27:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:08.430815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:08.466323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:08.530387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17049664-bb6e-4301-ae16-dac67047a12e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:08.563870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:08.564768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:08.565079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:08.565358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] PATH_INFO: `/v3/users/0fe481e3829f459fa524dd05e60d261b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:08.566549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] RBAC: Authorizing `identity:get_user(user_id=0fe481e3829f459fa524dd05e60d261b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:08.567982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:08.569651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 600/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 18 10:27:08 2022] GET /identity/v3/users/0fe481e3829f459fa524dd05e60d261b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:08.575846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:08.608114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:08.676867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd628c-6137-40b6-8472-b91303f84f68 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:08.711422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:08.712073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:08.712354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:08.712721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:08.781377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:08.782828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:08.813611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:08.880967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 598/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:27:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:27:08.887578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:08.987798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd76827-8373-42b5-a816-49a4af0ee7aa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:08.990133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:09.022252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c331a0-dd6c-4c96-ac82-3001de4f98a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:09.023869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c331a0-dd6c-4c96-ac82-3001de4f98a5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:09.024273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c331a0-dd6c-4c96-ac82-3001de4f98a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:09.024602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c331a0-dd6c-4c96-ac82-3001de4f98a5 None admin] PATH_INFO: `/v3/projects/b22934123a9447cc91ab604a5d2cea3c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:09.026095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c331a0-dd6c-4c96-ac82-3001de4f98a5 None admin] RBAC: Authorizing `identity:get_project(project_id=b22934123a9447cc91ab604a5d2cea3c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:09.027530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c331a0-dd6c-4c96-ac82-3001de4f98a5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:09.029261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 601/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 18 10:27:08 2022] GET /identity/v3/projects/b22934123a9447cc91ab604a5d2cea3c => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:09.035599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:09.133397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1054f53-07c5-4f65-b317-de5f1561e1ab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:09.162173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:09.167289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce73ad3-d127-490c-852c-21a70385cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:09.167940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73ad3-d127-490c-852c-21a70385cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:09.168223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73ad3-d127-490c-852c-21a70385cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:09.168498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73ad3-d127-490c-852c-21a70385cd2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:09.169351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce73ad3-d127-490c-852c-21a70385cd2c None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=b22934123a9447cc91ab604a5d2cea3c, user.id=0fe481e3829f459fa524dd05e60d261b) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:27:09.169780 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce73ad3-d127-490c-852c-21a70385cd2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:09.171102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce73ad3-d127-490c-852c-21a70385cd2c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:09.176948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 599/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:27:09 2022] GET /identity/v3/role_assignments?user.id=0fe481e3829f459fa524dd05e60d261b&scope.project.id=b22934123a9447cc91ab604a5d2cea3c&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:27:19.025385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=14 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:19.205420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:30.972205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365d0b-e3e6-4647-bff9-9f077931889a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:30.972830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365d0b-e3e6-4647-bff9-9f077931889a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:30.972830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365d0b-e3e6-4647-bff9-9f077931889a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:30.974824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 602/1201] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:30.980163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065b66-1739-4ee5-a036-696d1a3a2b02 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:30.980578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065b66-1739-4ee5-a036-696d1a3a2b02 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:30.980801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065b66-1739-4ee5-a036-696d1a3a2b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:30.998005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e0065b66-1739-4ee5-a036-696d1a3a2b02 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:30.999220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e0065b66-1739-4ee5-a036-696d1a3a2b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:31.065730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 600/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:31.090541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f365d0b-e3e6-4647-bff9-9f077931889a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:31.194433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f365d0b-e3e6-4647-bff9-9f077931889a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:31.229016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:31.229790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:31.230079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:31.230376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:31.241136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:31.242582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:31.259834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:31.261007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:31.262044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:31.262044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:31.262044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:31.262044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:31.262044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 603/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:31.267421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0065b66-1739-4ee5-a036-696d1a3a2b02 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:31.371771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0065b66-1739-4ee5-a036-696d1a3a2b02 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:31.400661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:31.406310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953c9a4-d169-4e2e-9f7d-28e3a088052e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:31.406972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953c9a4-d169-4e2e-9f7d-28e3a088052e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:31.407255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953c9a4-d169-4e2e-9f7d-28e3a088052e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:31.407530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953c9a4-d169-4e2e-9f7d-28e3a088052e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:31.408256 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953c9a4-d169-4e2e-9f7d-28e3a088052e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:31.408558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953c9a4-d169-4e2e-9f7d-28e3a088052e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:31.409983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953c9a4-d169-4e2e-9f7d-28e3a088052e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:31.421266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 601/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:31.427118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:31.480768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:31.529011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6942e-9eeb-40ca-9636-8d7bed50c7b6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:31.562822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe51557-b83f-4a8e-bb72-da2b12637329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:31.563494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe51557-b83f-4a8e-bb72-da2b12637329 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:31.563784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe51557-b83f-4a8e-bb72-da2b12637329 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:31.564197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe51557-b83f-4a8e-bb72-da2b12637329 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:31.564937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe51557-b83f-4a8e-bb72-da2b12637329 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:31.566408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe51557-b83f-4a8e-bb72-da2b12637329 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:31.744748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 604/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 18 10:28:31 2022] POST /identity/v3/users => generated 251 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:33.794332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007af16b-0eec-4392-9ca9-0cd758482770 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:33.794923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007af16b-0eec-4392-9ca9-0cd758482770 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:33.794923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007af16b-0eec-4392-9ca9-0cd758482770 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:33.796296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 602/1206] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:33.803350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d22dc6-9e2c-4777-aa47-adf71e687919 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:33.803350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d22dc6-9e2c-4777-aa47-adf71e687919 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:33.803350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d22dc6-9e2c-4777-aa47-adf71e687919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:33.824914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d8d22dc6-9e2c-4777-aa47-adf71e687919 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:33.826100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d8d22dc6-9e2c-4777-aa47-adf71e687919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:33.894528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 605/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:33.918074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007af16b-0eec-4392-9ca9-0cd758482770 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:34.024499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007af16b-0eec-4392-9ca9-0cd758482770 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:34.058160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:34.058871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:34.059152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:34.059427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:34.071116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:28:34.078690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:34.080267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:34.081290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 603/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:28:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:34.086296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d22dc6-9e2c-4777-aa47-adf71e687919 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:34.128392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:34.187881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d22dc6-9e2c-4777-aa47-adf71e687919 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:34.223250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:34.224082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:34.224431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:34.224731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:34.225548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:34.225858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:34.227258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:34.235399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 606/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:28:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:34.242573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:34.294340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:34.343522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce350ff-751f-4976-8481-113a845443ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:34.378607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:34.379495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:34.379805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:34.380079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:34.389863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:34.391317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:34.409129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:34.410469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:34.412360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:34.412360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:34.412360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:34.412360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:34.412360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 604/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:34.416867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:34.517301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee220e-57fb-465c-918b-eb9bc077baac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:34.550500 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:34.551153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:34.551432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:34.552483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:34.553463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:34.553772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:34.555130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:34.566472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 607/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:34.572422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:34.671192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830d30e-43e2-43cf-a6dd-39ee0971c04b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:34.700772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:34.706858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:34.709530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:34.709530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:34.709530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:34.717293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:34.718782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:28:34.733457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:34.734627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:34.735675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 605/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:34.740873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:34.804457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:34.839578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd78c-440c-4ec1-9e77-5e144c523f67 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:34.875602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:34.876276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:34.876560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:34.876867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:34.877708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:34.878014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:34.879444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:34.897002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 608/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 18 10:28:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:34.902962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:35.008865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f9bdd-eeae-45dc-a465-2ff617bee2a2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:35.044441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:35.045271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:35.045562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:35.045837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:35.121167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:35.122701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:35.207448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:35.220689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 606/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:28:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:28:35.226813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:35.246650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:35.329096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5e169-7a0e-4259-8bdc-e6a90e39e154 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:35.363382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:35.364246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:35.364540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:35.364838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:35.375457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:35.381817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:35.396756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:35.397783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:35.399048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:35.399048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:35.399048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:35.399048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:35.399048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 609/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:35.403728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:35.506147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e46c2-1e90-4f4f-8964-a665f01f5c71 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:35.516937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:35.541467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:35.542181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:35.542481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:35.542756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:35.543474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:35.543779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:35.545197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:35.557681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 607/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:35.563472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:35.663284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8597fb-af86-43a0-a050-ef89548027b6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:35.666466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:35.706833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:35.707843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:35.708298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:35.708575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:35.716697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:35.720608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:28:35.733835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:35.735008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:35.736365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 610/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:35.742315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:35.866029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96269-b84d-4c2d-9ba2-9c6d3792ad19 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:35.902034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2572bf5-5e79-4951-9bee-638508682c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:35.902782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572bf5-5e79-4951-9bee-638508682c50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:35.903063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572bf5-5e79-4951-9bee-638508682c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:35.903333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572bf5-5e79-4951-9bee-638508682c50 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:35.904122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2572bf5-5e79-4951-9bee-638508682c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:35.904427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2572bf5-5e79-4951-9bee-638508682c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:35.905897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2572bf5-5e79-4951-9bee-638508682c50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:35.918158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 608/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:28:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:35.924442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:35.964013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:36.032665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d581-eb4c-45e0-832e-3eeb05a52d9f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:36.060294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:36.065987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac2cdf-671c-4ed1-a59d-20543bd5686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:36.066638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac2cdf-671c-4ed1-a59d-20543bd5686f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:36.066922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac2cdf-671c-4ed1-a59d-20543bd5686f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:36.067688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac2cdf-671c-4ed1-a59d-20543bd5686f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:36.068515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac2cdf-671c-4ed1-a59d-20543bd5686f None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=8defe1e0d31249aa8bb27610229781b5) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:36.068864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac2cdf-671c-4ed1-a59d-20543bd5686f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:36.070294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac2cdf-671c-4ed1-a59d-20543bd5686f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:36.075721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 611/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:28:35 2022] GET /identity/v3/role_assignments?user.id=8defe1e0d31249aa8bb27610229781b5&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:38.069522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a4b10-02c1-45dc-a580-d2601ad5f455 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:38.070051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a4b10-02c1-45dc-a580-d2601ad5f455 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:38.070051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a4b10-02c1-45dc-a580-d2601ad5f455 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:38.071444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 609/1220] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:28:38.076689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d86b39-9c5a-46ac-913b-ea55a00b7a13 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:38.077121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d86b39-9c5a-46ac-913b-ea55a00b7a13 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:38.077283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d86b39-9c5a-46ac-913b-ea55a00b7a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:38.095306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-29d86b39-9c5a-46ac-913b-ea55a00b7a13 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:38.096513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-29d86b39-9c5a-46ac-913b-ea55a00b7a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:38.166845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 612/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:38.190228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a4b10-02c1-45dc-a580-d2601ad5f455 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:38.295944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a4b10-02c1-45dc-a580-d2601ad5f455 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:38.331151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:38.331882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:38.332159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:38.332429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:38.343220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:28:38.351281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:38.352425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:38.353664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 610/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:28:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:38.358519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d86b39-9c5a-46ac-913b-ea55a00b7a13 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:38.458768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d86b39-9c5a-46ac-913b-ea55a00b7a13 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:38.482516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:38.493096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:38.493768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:38.494053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:38.494327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:38.495072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:38.495379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:38.496756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:38.504665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 613/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:28:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:38.510511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:38.554126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:38.608811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535008e-100f-4fd5-a261-731a6eac552d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:38.641402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:38.642190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:38.642467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:38.642738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:38.653395 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:38.654848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:38.672630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:38.673770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:38.674847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:38.674847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:38.674847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:38.674847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:38.674847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 611/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:38.680395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:38.782396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fcd67-302f-4f39-a0e3-9d9e9429d8f3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:38.815910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:38.816575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:38.816888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:38.817660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:38.818525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:38.818830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:38.820206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:38.831785 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 614/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:38.838264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:38.934700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:38.941376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889287ef-28fc-4f51-bf52-0f50120feeff None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:38.976998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:38.977811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:38.978093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:38.978369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:38.985996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:38.987454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:28:39.002918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:39.005251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:39.007139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:39.008970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 612/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:39.011358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:39.062855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:39.117028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a2f87-ab9c-4f81-a0cb-fdab53bf7710 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:39.151529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:39.152190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:39.152473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:39.152746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:39.153660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:39.153962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:39.155408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:39.174080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 615/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 18 10:28:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:39.180587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:39.282165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ec8d1-0e5c-4bd1-ac7c-2a891c0047b7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:39.316577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:39.317996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:39.323432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:39.323432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:39.328686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:39.330159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:39.347148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:39.348253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:39.349434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:39.349434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:39.349434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:39.349434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:39.349434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 613/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:39.354353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:39.454844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb1073-60ab-48c0-9e5d-0b715e540625 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:39.457348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:39.488802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:39.489753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:39.490042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:39.490588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:39.491308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:39.491616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:39.493011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:39.504013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 616/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:39.510247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:39.522293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:39.612956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50904a-aad7-49ae-90fc-e981c6d5d539 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:39.647081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:39.647861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:39.648369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:39.648555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:39.654941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:39.656428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:28:39.669946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:39.676509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:39.677782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 614/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:28:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:39.678395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:39.782090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48803e9b-0ffc-4157-85e9-4aae234f3327 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:39.816996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cc599-f894-4acb-b843-b18fc4c01be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:39.817682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc599-f894-4acb-b843-b18fc4c01be1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:39.817971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc599-f894-4acb-b843-b18fc4c01be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:39.818254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc599-f894-4acb-b843-b18fc4c01be1 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:39.819056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cc599-f894-4acb-b843-b18fc4c01be1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:39.819399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cc599-f894-4acb-b843-b18fc4c01be1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:39.820835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cc599-f894-4acb-b843-b18fc4c01be1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:39.833795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 617/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:28:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:39.839837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:39.906501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:39.941606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448c1f0-7867-4cd3-996b-04577d24f7d5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:39.975741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5ae5-8326-4faf-a32a-baa3474a2753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:39.976631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5ae5-8326-4faf-a32a-baa3474a2753 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:39.976944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5ae5-8326-4faf-a32a-baa3474a2753 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:39.977219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5ae5-8326-4faf-a32a-baa3474a2753 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/8defe1e0d31249aa8bb27610229781b5/roles/30ebd4c6517a4a9ca23f5eaf73c732a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:39.991636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5ae5-8326-4faf-a32a-baa3474a2753 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=8defe1e0d31249aa8bb27610229781b5, role_id=30ebd4c6517a4a9ca23f5eaf73c732a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:39.993121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5ae5-8326-4faf-a32a-baa3474a2753 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:40.155161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 615/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:28:39 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/8defe1e0d31249aa8bb27610229781b5/roles/30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:42.185508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df71a-d969-42b0-b36d-2819818c83ab None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:42.186017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df71a-d969-42b0-b36d-2819818c83ab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:42.186115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df71a-d969-42b0-b36d-2819818c83ab None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:42.187485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 618/1233] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:42.193464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c824c-0609-4c8c-84e7-b7e35e425ccb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:42.193862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c824c-0609-4c8c-84e7-b7e35e425ccb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:42.194039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c824c-0609-4c8c-84e7-b7e35e425ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:42.211776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-618c824c-0609-4c8c-84e7-b7e35e425ccb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:42.213013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-618c824c-0609-4c8c-84e7-b7e35e425ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:42.283563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 616/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:42.307822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372df71a-d969-42b0-b36d-2819818c83ab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:42.334823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:42.422491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372df71a-d969-42b0-b36d-2819818c83ab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:42.461375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:42.462091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:42.462377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:42.462653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:42.473050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:28:42.480722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:42.481921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:42.483145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 619/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 18 10:28:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:42.488060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c824c-0609-4c8c-84e7-b7e35e425ccb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:42.538037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:42.591945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c824c-0609-4c8c-84e7-b7e35e425ccb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:42.625627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:42.626300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:42.626576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:42.626847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:42.627593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:42.627898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:42.630911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:42.638834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 617/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 18 10:28:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:42.644978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:42.733468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:42.749617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6caffdf-785b-450f-ad02-aad179ee25cd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:42.784730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:42.786106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:42.786391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:42.786663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:42.798087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:42.799622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:42.819787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:42.821190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:42.822109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:42.822109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:42.822109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:42.822109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:42.822109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 620/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:42.825874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:42.928683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b033a3-bf14-4a8c-a2b7-b0fd15f2ce6f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:42.938122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:42.965985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:42.966646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:42.966926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:42.967199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:42.967918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:42.968223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:42.969640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:42.981656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 618/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:42.987523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:43.093614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f42f9-9618-4da2-bec7-c1920ddee44b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:43.129864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:43.130672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:43.130963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:43.131243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:43.134323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:43.139922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:43.141424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:28:43.157012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:43.158166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:43.159365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 621/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:43.165274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:43.272372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa5aa0-40d9-471f-86a3-da70704750e0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:43.307649 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:43.308319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:43.308602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:43.308902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:43.309755 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:43.310050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:43.311612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:43.332227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 619/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 18 10:28:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:43.339142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:43.442510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479b740-c94f-4fec-bc58-d2dbed63b4d1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:43.476541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:43.477227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:43.477514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:43.477793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:43.544942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:43.551640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:43.553152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:43.663009 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 622/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:28:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:28:43.669079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:43.698045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:43.771127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6448e-6216-4646-83e9-97651a47401a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:43.810364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:43.810364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:43.810364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:43.810364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:43.816345 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:43.818499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:43.835586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:43.836636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:43.837764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:43.837764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:43.837764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:43.837764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:43.837764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 620/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:43.842530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:43.945075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:43.951177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3af99-ff3e-40e9-9da2-134f9fb377f1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:43.985145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:43.985807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:43.986087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:43.986361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:43.987079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:43.987384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:43.989462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:44.001805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 623/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:44.007726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:44.089624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:44.111205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed013c7-08c5-4d46-af22-448e14d8d456 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:44.146938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:44.147739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:44.148026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:44.148305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:44.155113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:44.156679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:28:44.170574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:44.171642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:44.172805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 621/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:28:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:44.178071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:44.293807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a820b-385f-4d94-904f-1934f2597ed3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:44.328222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082af6-1b20-4dfe-b6d2-2f862a38e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:44.328930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082af6-1b20-4dfe-b6d2-2f862a38e736 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:44.329220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082af6-1b20-4dfe-b6d2-2f862a38e736 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:44.329499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082af6-1b20-4dfe-b6d2-2f862a38e736 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:44.330296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca082af6-1b20-4dfe-b6d2-2f862a38e736 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:44.330607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca082af6-1b20-4dfe-b6d2-2f862a38e736 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:44.332108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca082af6-1b20-4dfe-b6d2-2f862a38e736 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:44.345510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 624/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:28:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:44.351973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:44.458267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77934ba2-53d8-485b-ae9c-43738a7b4fb2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:44.497581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9460031-d27b-403c-9a1a-bcedb04898b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:44.498222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9460031-d27b-403c-9a1a-bcedb04898b0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:44.498512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9460031-d27b-403c-9a1a-bcedb04898b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:44.498792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9460031-d27b-403c-9a1a-bcedb04898b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:44.499629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9460031-d27b-403c-9a1a-bcedb04898b0 None admin] RBAC: Adding query filter params (role.id=30ebd4c6517a4a9ca23f5eaf73c732a0, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=8defe1e0d31249aa8bb27610229781b5) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:44.499937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9460031-d27b-403c-9a1a-bcedb04898b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:44.501479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9460031-d27b-403c-9a1a-bcedb04898b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:44.514198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 622/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:28:44 2022] GET /identity/v3/role_assignments?user.id=8defe1e0d31249aa8bb27610229781b5&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=30ebd4c6517a4a9ca23f5eaf73c732a0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:46.647530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4c87c-14dc-4044-9234-2982d7271c0d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:46.648175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4c87c-14dc-4044-9234-2982d7271c0d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:46.648175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4c87c-14dc-4044-9234-2982d7271c0d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:46.649532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 625/1247] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:46.654819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3ce9b-1b70-4574-a069-632c845cc0c4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:46.655220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3ce9b-1b70-4574-a069-632c845cc0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:46.655387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3ce9b-1b70-4574-a069-632c845cc0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:46.660977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:46.673517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15c3ce9b-1b70-4574-a069-632c845cc0c4 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:46.674704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15c3ce9b-1b70-4574-a069-632c845cc0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:46.743876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 623/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:46.767545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4c87c-14dc-4044-9234-2982d7271c0d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:46.786977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:46.879236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4c87c-14dc-4044-9234-2982d7271c0d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:46.915328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:46.916047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:46.916330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:46.916603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:46.927642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:28:46.935185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:46.936379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:46.937484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 626/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:28:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:46.943280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3ce9b-1b70-4574-a069-632c845cc0c4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:47.040066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3ce9b-1b70-4574-a069-632c845cc0c4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:47.069677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:47.074744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:47.075422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:47.075704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:47.075978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:47.076729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:47.077067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:47.078460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:47.087223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 624/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:28:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:47.093104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:47.172886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:47.195740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8fb19-1d63-4e7a-ad98-f72dab3a5d02 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:47.229163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:47.229954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:47.230233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:47.230507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:47.241852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:47.243289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:47.260106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:47.261245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:47.262349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:47.262349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:47.262349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:47.262349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:47.262349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 627/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:47.267214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:47.374508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab80a2-1fff-4d27-8a51-7cbba21a1b95 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:47.407972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:47.408675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:47.408995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:47.420425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:47.420425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:47.420425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:47.420425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:47.424419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 625/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:47.431190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:47.535029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b710c5a-caa1-4497-949f-4f44cec04812 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:47.563683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:47.571919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:47.572924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:47.573136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:47.573425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:47.581469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:47.584349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:28:47.598030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:47.600079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:47.601422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 628/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:47.606996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:47.666365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:47.715011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168badb-823e-4d2f-8480-50a96b28ca15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:47.751790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:47.752462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:47.752751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:47.753234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:47.754092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:47.754653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:47.756360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:47.775058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 626/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 18 10:28:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:28:47.781095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:47.885513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf171-798c-4940-bf5c-4f095aba47f7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:47.919551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:47.920204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:47.920492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:47.920800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:47.980320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:47.994402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:47.995864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.097304 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 629/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:28:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:28:48.103938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.204440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afbde-8eb7-40c6-81e4-6afe95100ee5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.218181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:48.239778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.240761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.241073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.241347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.251231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.252708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:48.270829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:48.272142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:48.273386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:48.273386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:48.273386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:48.273386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:48.273386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 627/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:48.278065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.373288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:48.385283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555025b-d943-4c11-b0ff-b176eabd0863 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.418967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.419626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.419913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.420187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.420948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:48.421249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.422616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.434210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 630/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:48.439942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.541759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd82ecc-7252-4c08-9cb3-5bde179c9a93 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.575172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.576033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.576317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.576593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.583886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.585384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:28:48.598733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:48.605077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:48.606200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 628/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:48.606827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.708596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786533-7580-4f3f-8914-90d91a2aed9d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.742256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05d4d9-59da-431c-80dd-65312597ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.742918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05d4d9-59da-431c-80dd-65312597ed9c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.743201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05d4d9-59da-431c-80dd-65312597ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.743480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05d4d9-59da-431c-80dd-65312597ed9c None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.744268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05d4d9-59da-431c-80dd-65312597ed9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:48.744583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05d4d9-59da-431c-80dd-65312597ed9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.746034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05d4d9-59da-431c-80dd-65312597ed9c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.758791 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 631/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:48.765192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.791196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:48.864794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d627fa-3ecd-4b0e-bba3-d791c1e85867 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.898826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a9146-9634-44d5-9a09-2b96de9267f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.899486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a9146-9634-44d5-9a09-2b96de9267f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.899770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a9146-9634-44d5-9a09-2b96de9267f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.900047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a9146-9634-44d5-9a09-2b96de9267f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.900896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a9146-9634-44d5-9a09-2b96de9267f5 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=8defe1e0d31249aa8bb27610229781b5) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:48.901210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a9146-9634-44d5-9a09-2b96de9267f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.902665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a9146-9634-44d5-9a09-2b96de9267f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.908115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 629/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/role_assignments?user.id=8defe1e0d31249aa8bb27610229781b5&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:50.911198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0267ce-d2a6-4186-88bc-cfff7bea36e2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.911747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0267ce-d2a6-4186-88bc-cfff7bea36e2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.911747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0267ce-d2a6-4186-88bc-cfff7bea36e2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.913185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 632/1261] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:50.918637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b19a1-796e-4cf9-888e-dc55e4d0fcf1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.919114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b19a1-796e-4cf9-888e-dc55e4d0fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.919214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b19a1-796e-4cf9-888e-dc55e4d0fcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.937022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1d8b19a1-796e-4cf9-888e-dc55e4d0fcf1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:50.938197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1d8b19a1-796e-4cf9-888e-dc55e4d0fcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:51.010003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 630/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:51.033310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0267ce-d2a6-4186-88bc-cfff7bea36e2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.044766 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.137459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0267ce-d2a6-4186-88bc-cfff7bea36e2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.172330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.173152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.173435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.173704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.185237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:28:51.193943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:51.195044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:51.196290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 633/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:51.201267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b19a1-796e-4cf9-888e-dc55e4d0fcf1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.219565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.300641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b19a1-796e-4cf9-888e-dc55e4d0fcf1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.335079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.335772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.336080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.336351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.337121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:51.337423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.338802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.346946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 631/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:51.352938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.421435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.455613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955445b4-fabf-4d15-a1fb-3fd0fe3bdb11 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.488919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.489701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.490309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.490597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.501877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.503439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:51.520842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:51.521948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:51.523054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:51.523054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:51.523054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:51.523054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:51.523054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 634/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:51.527982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.610093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.630152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c3790-8b32-47db-9fdf-03f1b7bae293 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.663342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.664006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.664289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.664566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.665314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:51.665622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.667015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.679891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 632/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:51.686270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.786308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e0c29-0e9f-4233-b543-7738d109c111 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.806080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.820144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.820984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.821271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.821548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.829080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.830516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:28:51.844455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:51.845606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:51.846764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 635/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:51.852227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.950628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273cbcf-41c7-4202-88e7-b9c5aefd8d0e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.987734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.988823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.989089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.989493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.990724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:51.991029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.992462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.001556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:52.010552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 633/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:52.016559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.120039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b35b9-0aa1-4ee0-a4a4-dcef2da4ee67 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.153027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.153824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.154105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.154383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.165804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.167246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:52.184213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:52.185317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:52.186272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:52.186272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:52.186272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:52.186272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:52.186272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 636/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:52.191587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.292147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c50e1f-5869-4f8a-a4f7-657968040715 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.326639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.333687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.333687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.333687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.333687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:52.333687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.333687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.342382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 634/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:52.348782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.380761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:52.449517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda571a-793b-4900-866f-124bfc93ba14 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.482950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.483737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.484020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.484296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.491545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.493017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:28:52.507883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:52.508972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:52.509915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 637/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:28:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:52.515236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.575046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:52.617031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30838355-dd19-4c13-936c-1d801c273b4e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.651686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c4ae-c089-4458-b1c2-e45caff044cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.652342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c4ae-c089-4458-b1c2-e45caff044cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.652627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c4ae-c089-4458-b1c2-e45caff044cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.652930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c4ae-c089-4458-b1c2-e45caff044cb None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.653715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c4ae-c089-4458-b1c2-e45caff044cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:52.654023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c4ae-c089-4458-b1c2-e45caff044cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.655465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c4ae-c089-4458-b1c2-e45caff044cb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.667406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 635/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:28:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:52.673685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.770495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:52.779314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdcf4f-15c0-46f8-9014-39e080f3b47a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.814416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb72bf-bc8a-41e2-a3af-065f5ce11804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.815293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb72bf-bc8a-41e2-a3af-065f5ce11804 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.815574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb72bf-bc8a-41e2-a3af-065f5ce11804 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.815847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb72bf-bc8a-41e2-a3af-065f5ce11804 None admin] PATH_INFO: `/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/8defe1e0d31249aa8bb27610229781b5/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.818478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb72bf-bc8a-41e2-a3af-065f5ce11804 None admin] RBAC: Authorizing `identity:create_grant(project_id=2509fcdb62684f97a2842ce68f7081fc, user_id=8defe1e0d31249aa8bb27610229781b5, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.819952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb72bf-bc8a-41e2-a3af-065f5ce11804 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.900478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 638/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 18 10:28:52 2022] PUT /identity/v3/projects/2509fcdb62684f97a2842ce68f7081fc/users/8defe1e0d31249aa8bb27610229781b5/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:28:54.894728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1511f-4254-40a0-912f-a8b34a1e05a2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.895274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1511f-4254-40a0-912f-a8b34a1e05a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.895274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1511f-4254-40a0-912f-a8b34a1e05a2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.896647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 636/1274] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:54.902132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29011ab-38ac-468b-9fd4-e0739a208dab None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.902522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29011ab-38ac-468b-9fd4-e0739a208dab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.902707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29011ab-38ac-468b-9fd4-e0739a208dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.922367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f29011ab-38ac-468b-9fd4-e0739a208dab None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:54.923787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f29011ab-38ac-468b-9fd4-e0739a208dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:54.995452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 639/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:55.019173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1511f-4254-40a0-912f-a8b34a1e05a2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:55.125888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1511f-4254-40a0-912f-a8b34a1e05a2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:55.143784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:55.162911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:55.163733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:55.164018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:55.164329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:55.176094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:28:55.184034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:55.185250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:55.190824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 637/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 18 10:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:55.191629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29011ab-38ac-468b-9fd4-e0739a208dab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:55.274611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:55.293282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29011ab-38ac-468b-9fd4-e0739a208dab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:55.328830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:55.329503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:55.329815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:55.330093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:55.330882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:55.331190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:55.334016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:55.342293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 640/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 18 10:28:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:28:55.348588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:55.455158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8d2ea-adef-4c36-957c-e410744e9c87 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:55.491106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:55.491922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:55.492219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:55.492487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:55.509142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:55.511818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:55.537413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:55.538602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:55.539640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:55.539640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:55.539640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:55.539640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:55.539640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 638/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:55.544926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:55.648186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb88634-8ce4-43a4-8966-4a8e8e78548c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:55.682220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:55.682884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:55.683172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:55.683452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:55.684178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:55.684489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:55.685924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:55.698298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 641/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:55.704169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:55.737091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:55.818249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb87122-5c00-4ad1-ac70-b0864f45d1dd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:55.855225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:55.856020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:55.856297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:55.856570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:55.865995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:55.867509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:28:55.883528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:55.884745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:55.886244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 639/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:55.891361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:55.896057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:56.001651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccc453-d1f4-49e6-b9fc-e41c709b03d0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:56.042291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:56.042973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:56.043250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:56.043644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:56.044761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:56.045285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:56.047764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:56.067545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 642/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 18 10:28:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:56.073649 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:56.173226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:56.184427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8dd79-5a6b-4d8b-b137-39e77fa1559d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:56.222258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:56.223498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:56.224002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:56.224393 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:56.296632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:56.298042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:56.404792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:56.418891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 640/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 18 10:28:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:28:56.424756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:56.490490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:56.528284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d24162-6ad9-4474-9e68-b7f12d48b14c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:56.562848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:56.563569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:56.563855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:56.564132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:56.574522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:56.575960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:28:56.594829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:56.596152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:56.597237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:28:56.597237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:28:56.597237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:28:56.597237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:56.597237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 643/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:56.602901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:56.721864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb7daa-c32d-4236-852f-fd104840b273 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:56.762778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:56.763439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:56.763721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:56.763991 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:56.764814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:56.765116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:56.766508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:56.779429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 641/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 18 10:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:56.786445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:56.903670 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce64809-956a-44b7-87e0-95652546baeb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:56.918772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:56.940487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:56.941949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:56.942801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:56.943081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:56.950324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:56.951828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:28:56.967162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:56.968306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 18 10:28:56.969664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 644/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 18 10:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:28:56.974540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:57.030403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:57.084540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8def4ab-fba1-492e-9a55-04db58e124fe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:57.120318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bebfc9-2c67-4d97-aa1b-7310fb103db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:57.121009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bebfc9-2c67-4d97-aa1b-7310fb103db2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:57.121298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bebfc9-2c67-4d97-aa1b-7310fb103db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:57.121569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bebfc9-2c67-4d97-aa1b-7310fb103db2 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:57.122357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bebfc9-2c67-4d97-aa1b-7310fb103db2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:57.122664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bebfc9-2c67-4d97-aa1b-7310fb103db2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:57.124138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bebfc9-2c67-4d97-aa1b-7310fb103db2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:57.139459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 642/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 18 10:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:57.146642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:57.255195 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e4f3d-8f98-4e74-ba5a-767fb6dd2456 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:57.291694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b98963-0519-4045-b0b1-da4b8d633668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:57.292394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b98963-0519-4045-b0b1-da4b8d633668 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:57.292679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b98963-0519-4045-b0b1-da4b8d633668 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:57.292985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b98963-0519-4045-b0b1-da4b8d633668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:57.293829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b98963-0519-4045-b0b1-da4b8d633668 None admin] RBAC: Adding query filter params (role.id=fa0d2e27b7af41708eee88eac77dbebf, scope.project.id=2509fcdb62684f97a2842ce68f7081fc, user.id=8defe1e0d31249aa8bb27610229781b5) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:57.294144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b98963-0519-4045-b0b1-da4b8d633668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:57.295576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b98963-0519-4045-b0b1-da4b8d633668 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:57.302685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 645/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 18 10:28:57 2022] GET /identity/v3/role_assignments?user.id=8defe1e0d31249aa8bb27610229781b5&scope.project.id=2509fcdb62684f97a2842ce68f7081fc&role.id=fa0d2e27b7af41708eee88eac77dbebf => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:28:59.420624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335e34a-4e58-404e-b4ef-1891b5943782 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:59.421222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335e34a-4e58-404e-b4ef-1891b5943782 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:59.421320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335e34a-4e58-404e-b4ef-1891b5943782 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:59.422716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 643/1288] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:59.428896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19f1b1-5038-4191-97c9-15c4d600a225 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:59.429634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19f1b1-5038-4191-97c9-15c4d600a225 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:59.429634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19f1b1-5038-4191-97c9-15c4d600a225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:59.446910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6a19f1b1-5038-4191-97c9-15c4d600a225 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:59.448190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a19f1b1-5038-4191-97c9-15c4d600a225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:59.480205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:59.519337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 646/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:28:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:28:59.541329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335e34a-4e58-404e-b4ef-1891b5943782 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:59.654521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335e34a-4e58-404e-b4ef-1891b5943782 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:59.668583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:59.695875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:59.698034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:59.698034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:59.698034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:59.698034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:59.699335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:28:59.712231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:28:59.723081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:28:59.725005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 644/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:28:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:28:59.726113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19f1b1-5038-4191-97c9-15c4d600a225 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:59.829240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19f1b1-5038-4191-97c9-15c4d600a225 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:59.868906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01719aaf-f99f-4176-ac41-9b1540e35b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:59.869592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01719aaf-f99f-4176-ac41-9b1540e35b74 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:59.869881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01719aaf-f99f-4176-ac41-9b1540e35b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:59.870159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01719aaf-f99f-4176-ac41-9b1540e35b74 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:59.870886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01719aaf-f99f-4176-ac41-9b1540e35b74 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:59.871192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01719aaf-f99f-4176-ac41-9b1540e35b74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:59.872590 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01719aaf-f99f-4176-ac41-9b1540e35b74 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:59.880665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 647/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 18 10:28:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:59.886810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:59.997536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2741c4-f86c-48d3-ae6b-5da8cb2b80d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.033639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7f19-3b0c-4b66-bb24-ebd2502535a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.034324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7f19-3b0c-4b66-bb24-ebd2502535a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.034613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7f19-3b0c-4b66-bb24-ebd2502535a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.035260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7f19-3b0c-4b66-bb24-ebd2502535a6 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.035779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7f19-3b0c-4b66-bb24-ebd2502535a6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:00.035939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7f19-3b0c-4b66-bb24-ebd2502535a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:00.037444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7f19-3b0c-4b66-bb24-ebd2502535a6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:00.042489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 645/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 18 10:28:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:02.083167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2ff9b-8b84-473c-9903-c97b29173d30 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:02.083674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2ff9b-8b84-473c-9903-c97b29173d30 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:02.083674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2ff9b-8b84-473c-9903-c97b29173d30 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:02.085176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 648/1293] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:29:02.090777 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4082e25-86c4-419e-bb50-b8a9cb7b5c84 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:02.091216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4082e25-86c4-419e-bb50-b8a9cb7b5c84 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:02.091362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4082e25-86c4-419e-bb50-b8a9cb7b5c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:02.112539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b4082e25-86c4-419e-bb50-b8a9cb7b5c84 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:02.114167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4082e25-86c4-419e-bb50-b8a9cb7b5c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:02.148291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:02.192515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 646/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:29:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:29:02.216161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2ff9b-8b84-473c-9903-c97b29173d30 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:02.259171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:02.342963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2ff9b-8b84-473c-9903-c97b29173d30 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:02.383058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299a593-3ffb-4f0a-a63a-03c687995086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:02.383786 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299a593-3ffb-4f0a-a63a-03c687995086 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:02.384072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299a593-3ffb-4f0a-a63a-03c687995086 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:02.384352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299a593-3ffb-4f0a-a63a-03c687995086 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:02.385131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299a593-3ffb-4f0a-a63a-03c687995086 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:02.392948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299a593-3ffb-4f0a-a63a-03c687995086 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:02.633012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 649/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 18 10:29:02 2022] POST /identity/v3/services => generated 246 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:04.692701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014aa7d-8f6c-4001-827a-fbc38c36271a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:04.693295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014aa7d-8f6c-4001-827a-fbc38c36271a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:04.693295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014aa7d-8f6c-4001-827a-fbc38c36271a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:04.695180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 647/1296] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:29:04.700627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cccb54-7a65-437b-8cfb-d3094ce10933 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:04.701063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cccb54-7a65-437b-8cfb-d3094ce10933 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:04.701210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cccb54-7a65-437b-8cfb-d3094ce10933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:04.718715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-45cccb54-7a65-437b-8cfb-d3094ce10933 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:04.719900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-45cccb54-7a65-437b-8cfb-d3094ce10933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:04.793964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 650/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:29:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:29:04.818245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014aa7d-8f6c-4001-827a-fbc38c36271a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:04.924312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014aa7d-8f6c-4001-827a-fbc38c36271a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:04.966428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:04.966428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:04.966712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:04.966813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:04.967469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:04.968890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:29:04.980909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:29:04.982177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:29:04.983142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 648/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:29:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:29:04.988875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cccb54-7a65-437b-8cfb-d3094ce10933 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:05.008229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:05.097938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cccb54-7a65-437b-8cfb-d3094ce10933 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:05.151551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:05.153383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:05.154101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:05.154853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:05.157275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:05.158107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:05.162292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:05.176712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 651/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 18 10:29:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:05.186248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:05.225177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:05.294119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156ddb8-135e-4d0b-8a80-c46e60ee3a93 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:05.330300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c9f0c-1148-47dc-9937-5712e22aecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:05.330958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c9f0c-1148-47dc-9937-5712e22aecc1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:05.331257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c9f0c-1148-47dc-9937-5712e22aecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:05.331512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c9f0c-1148-47dc-9937-5712e22aecc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:05.332625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c9f0c-1148-47dc-9937-5712e22aecc1 None admin] RBAC: Adding query filter params (interface=public, service_id=825bc792a5fd42cebadf530f1801cc45, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:05.332967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c9f0c-1148-47dc-9937-5712e22aecc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:05.334361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c9f0c-1148-47dc-9937-5712e22aecc1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:05.339463 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 649/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 18 10:29:05 2022] GET /identity/v3/endpoints?service_id=825bc792a5fd42cebadf530f1801cc45&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:05.345367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:05.435749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:05.453067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62362e-859c-46ad-84c1-f7f667ae7bd1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:05.486216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e71aea-b0da-480e-b9c2-7d5ca0ad44cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:05.486800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e71aea-b0da-480e-b9c2-7d5ca0ad44cc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:05.487084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e71aea-b0da-480e-b9c2-7d5ca0ad44cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:05.487357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e71aea-b0da-480e-b9c2-7d5ca0ad44cc None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:05.488856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e71aea-b0da-480e-b9c2-7d5ca0ad44cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:05.490526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e71aea-b0da-480e-b9c2-7d5ca0ad44cc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:05.498032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 652/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 18 10:29:05 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:07.609369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c091a-1816-4fe2-a601-cde197bdf324 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.609970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c091a-1816-4fe2-a601-cde197bdf324 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.609970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c091a-1816-4fe2-a601-cde197bdf324 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.611330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 650/1302] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:29:07.618296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06cc51-59f2-453d-a644-18fbf51bf720 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.619020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06cc51-59f2-453d-a644-18fbf51bf720 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.619020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06cc51-59f2-453d-a644-18fbf51bf720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.637130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1b06cc51-59f2-453d-a644-18fbf51bf720 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:07.638414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b06cc51-59f2-453d-a644-18fbf51bf720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:07.719001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 653/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:29:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:29:07.743950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c091a-1816-4fe2-a601-cde197bdf324 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:07.857448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:07.865775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c091a-1816-4fe2-a601-cde197bdf324 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:07.915324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:07.916222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.916513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.916864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.917538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:07.919034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:29:07.932183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:29:07.933411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jul 18 10:29:07.934281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 651/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 18 10:29:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:29:07.940191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06cc51-59f2-453d-a644-18fbf51bf720 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:08.044180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:08.071852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06cc51-59f2-453d-a644-18fbf51bf720 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:08.114221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb009a-bd47-42e0-9482-e99da5cb1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:08.115401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb009a-bd47-42e0-9482-e99da5cb1641 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:08.115694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb009a-bd47-42e0-9482-e99da5cb1641 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:08.116022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb009a-bd47-42e0-9482-e99da5cb1641 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:08.116737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb009a-bd47-42e0-9482-e99da5cb1641 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:08.117085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb009a-bd47-42e0-9482-e99da5cb1641 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:08.118663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb009a-bd47-42e0-9482-e99da5cb1641 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:08.124996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 654/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 18 10:29:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:08.132539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:08.266279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199eef1-faa4-4c03-ab22-61863fc88f94 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:08.314512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b60c6-ce82-4d19-9d6f-3a1b372cdf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:08.315492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b60c6-ce82-4d19-9d6f-3a1b372cdf7e None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:08.315769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b60c6-ce82-4d19-9d6f-3a1b372cdf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:08.316370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b60c6-ce82-4d19-9d6f-3a1b372cdf7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:08.317205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b60c6-ce82-4d19-9d6f-3a1b372cdf7e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:08.318624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b60c6-ce82-4d19-9d6f-3a1b372cdf7e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:08.441640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 652/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 18 10:29:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:18.192610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=30 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:18.436017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:07.750083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a35f3-af9a-4603-b385-d8c3b4103618 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:07.750557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a35f3-af9a-4603-b385-d8c3b4103618 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:07.750657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a35f3-af9a-4603-b385-d8c3b4103618 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:07.758038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabc5a3-b368-45fe-9807-e118f1d7b989 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:07.758566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabc5a3-b368-45fe-9807-e118f1d7b989 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:07.758678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabc5a3-b368-45fe-9807-e118f1d7b989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:07.761014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 655/1307] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:07.845144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2eabc5a3-b368-45fe-9807-e118f1d7b989 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:07.846321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2eabc5a3-b368-45fe-9807-e118f1d7b989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:07.955196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 653/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:37:08.006589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaf3f8-1ba0-4f80-bf2e-2c4269cbea1b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:08.007554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaf3f8-1ba0-4f80-bf2e-2c4269cbea1b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:08.007992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaf3f8-1ba0-4f80-bf2e-2c4269cbea1b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:08.009936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 656/1309] 10.222.0.49 () {54 vars in 965 bytes} [Mon Jul 18 10:37:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:08.017145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ea76-4cfe-4eae-93fc-ae162e017eb3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:08.017554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ea76-4cfe-4eae-93fc-ae162e017eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:08.017899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ea76-4cfe-4eae-93fc-ae162e017eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:08.124557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-54c1ea76-4cfe-4eae-93fc-ae162e017eb3 None None] MFA Rules not processed for user `a48b0fdacb2c43649916304e1237f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:08.125885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54c1ea76-4cfe-4eae-93fc-ae162e017eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:08.150137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:08.213361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 654/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Mon Jul 18 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:08.222918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbaf3f8-1ba0-4f80-bf2e-2c4269cbea1b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:08.224748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5fbaf3f8-1ba0-4f80-bf2e-2c4269cbea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:08.353101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbaf3f8-1ba0-4f80-bf2e-2c4269cbea1b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:08.451556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972226d-e2ef-4c47-97e2-29987dbf794f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:08.452222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972226d-e2ef-4c47-97e2-29987dbf794f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:08.452506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972226d-e2ef-4c47-97e2-29987dbf794f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:08.452805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972226d-e2ef-4c47-97e2-29987dbf794f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:08.454624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0972226d-e2ef-4c47-97e2-29987dbf794f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:08.527183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:08.592396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972226d-e2ef-4c47-97e2-29987dbf794f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:08.593912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972226d-e2ef-4c47-97e2-29987dbf794f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:08.617801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 657/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:10.985352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2e160-864d-4010-a08b-2e71707bd32f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:10.985878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2e160-864d-4010-a08b-2e71707bd32f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:10.986075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2e160-864d-4010-a08b-2e71707bd32f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:10.987450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 655/1312] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:10.994735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d30e9f-3c3f-4907-9def-15c481094bae None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:10.997272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d30e9f-3c3f-4907-9def-15c481094bae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:10.997272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d30e9f-3c3f-4907-9def-15c481094bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:11.092496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b9d30e9f-3c3f-4907-9def-15c481094bae None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:11.093711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b9d30e9f-3c3f-4907-9def-15c481094bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:11.219724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 658/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:11.261311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2e160-864d-4010-a08b-2e71707bd32f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:11.263026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e0c2e160-864d-4010-a08b-2e71707bd32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:11.386989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2e160-864d-4010-a08b-2e71707bd32f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:11.472240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:11.475152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ec2cc-aecf-45ab-a1f1-82560a68bdb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:11.476503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec2cc-aecf-45ab-a1f1-82560a68bdb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:11.476891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec2cc-aecf-45ab-a1f1-82560a68bdb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:11.477234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec2cc-aecf-45ab-a1f1-82560a68bdb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:11.479204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-028ec2cc-aecf-45ab-a1f1-82560a68bdb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:11.604016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ec2cc-aecf-45ab-a1f1-82560a68bdb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:11.605776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ec2cc-aecf-45ab-a1f1-82560a68bdb1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:11.624910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 656/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:14.231361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bade1-47d3-4410-b043-302bacfc2a3e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:14.231953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bade1-47d3-4410-b043-302bacfc2a3e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:14.231953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bade1-47d3-4410-b043-302bacfc2a3e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:14.233362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 659/1315] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:14.238842 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d41756-2b49-41d6-85ee-4641241ce63d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:14.239298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d41756-2b49-41d6-85ee-4641241ce63d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:14.239391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d41756-2b49-41d6-85ee-4641241ce63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:14.306771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b5d41756-2b49-41d6-85ee-4641241ce63d None None] MFA Rules not processed for user `d024a945bdaf4746aa6cddd29acc851e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:14.308034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5d41756-2b49-41d6-85ee-4641241ce63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:14.365800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:14.412601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 657/1316] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:14.453247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bade1-47d3-4410-b043-302bacfc2a3e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:14.454768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-193bade1-47d3-4410-b043-302bacfc2a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:14.464933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:14.564161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bade1-47d3-4410-b043-302bacfc2a3e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:14.640583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538a251-68dd-45f8-902b-e2d999aff59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:14.641245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538a251-68dd-45f8-902b-e2d999aff59f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:14.641536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538a251-68dd-45f8-902b-e2d999aff59f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:14.641812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538a251-68dd-45f8-902b-e2d999aff59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:14.643472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2538a251-68dd-45f8-902b-e2d999aff59f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:14.708979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:14.759810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538a251-68dd-45f8-902b-e2d999aff59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:14.761332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538a251-68dd-45f8-902b-e2d999aff59f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:14.781942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 660/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:17.841673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5e4f-caee-4e0a-94c7-16f7eff2192c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:17.842500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5e4f-caee-4e0a-94c7-16f7eff2192c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:17.842660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5e4f-caee-4e0a-94c7-16f7eff2192c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:17.844077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 658/1318] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:17.850039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852733a6-4165-4366-a62e-019da0fba052 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:17.850601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852733a6-4165-4366-a62e-019da0fba052 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:17.850697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852733a6-4165-4366-a62e-019da0fba052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:17.922877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-852733a6-4165-4366-a62e-019da0fba052 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:17.924137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-852733a6-4165-4366-a62e-019da0fba052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:18.029535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 661/1319] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:18.086426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c2464-7cb1-4c27-bbf0-baebda6ad5fb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:18.086725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c2464-7cb1-4c27-bbf0-baebda6ad5fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:18.086999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c2464-7cb1-4c27-bbf0-baebda6ad5fb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:18.088373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 659/1320] 10.222.0.49 () {54 vars in 965 bytes} [Mon Jul 18 10:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:18.094824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef4c4d-a878-4425-ac24-6b7acaecaaed None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:18.095114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef4c4d-a878-4425-ac24-6b7acaecaaed None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:18.095423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef4c4d-a878-4425-ac24-6b7acaecaaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:18.117940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:18.179770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dcef4c4d-a878-4425-ac24-6b7acaecaaed None None] MFA Rules not processed for user `a48b0fdacb2c43649916304e1237f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:18.181034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dcef4c4d-a878-4425-ac24-6b7acaecaaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:18.264553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 662/1321] 10.222.0.49 () {58 vars in 1062 bytes} [Mon Jul 18 10:37:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:18.274272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c2464-7cb1-4c27-bbf0-baebda6ad5fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:18.276302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b34c2464-7cb1-4c27-bbf0-baebda6ad5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:18.386708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c2464-7cb1-4c27-bbf0-baebda6ad5fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:18.467102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77e1b3-4e87-45b8-92f5-ba236479e41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:18.467752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77e1b3-4e87-45b8-92f5-ba236479e41a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:18.468035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77e1b3-4e87-45b8-92f5-ba236479e41a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:18.468308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77e1b3-4e87-45b8-92f5-ba236479e41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:18.469994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f77e1b3-4e87-45b8-92f5-ba236479e41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:18.478069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:18.582311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77e1b3-4e87-45b8-92f5-ba236479e41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:18.583775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77e1b3-4e87-45b8-92f5-ba236479e41a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:18.602855 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 660/1322] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:37:22.259970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd264fb2-1864-431e-872f-9f0eb9709667 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:22.260802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd264fb2-1864-431e-872f-9f0eb9709667 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:22.260802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd264fb2-1864-431e-872f-9f0eb9709667 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:22.262232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 663/1323] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:22.268151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b8d2-9e55-40ca-a59f-fc52ecd656b7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:22.268857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b8d2-9e55-40ca-a59f-fc52ecd656b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:22.268857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b8d2-9e55-40ca-a59f-fc52ecd656b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:22.344209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-50d1b8d2-9e55-40ca-a59f-fc52ecd656b7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:22.345476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50d1b8d2-9e55-40ca-a59f-fc52ecd656b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:22.406298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:22.447864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 661/1324] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:22.485332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd264fb2-1864-431e-872f-9f0eb9709667 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:22.486875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd264fb2-1864-431e-872f-9f0eb9709667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:22.604744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd264fb2-1864-431e-872f-9f0eb9709667 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:22.619821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:22.690464 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9be0f-1d47-4a42-bfd3-6d24ccf91542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:22.691424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9be0f-1d47-4a42-bfd3-6d24ccf91542 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:22.691544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9be0f-1d47-4a42-bfd3-6d24ccf91542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:22.691759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9be0f-1d47-4a42-bfd3-6d24ccf91542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:22.693608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7e9be0f-1d47-4a42-bfd3-6d24ccf91542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:22.822114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9be0f-1d47-4a42-bfd3-6d24ccf91542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:22.823559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9be0f-1d47-4a42-bfd3-6d24ccf91542 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:22.842815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 664/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:24.978393 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967f4d-2ac0-4b9d-b9fc-c1283ec604a3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:24.978932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967f4d-2ac0-4b9d-b9fc-c1283ec604a3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:24.978932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967f4d-2ac0-4b9d-b9fc-c1283ec604a3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:24.980567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 662/1326] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:24.986137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0fb02-8c5f-48f0-a33d-fed3e9e5aab8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:24.986647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0fb02-8c5f-48f0-a33d-fed3e9e5aab8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:24.986746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0fb02-8c5f-48f0-a33d-fed3e9e5aab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:25.038493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:25.059550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dad0fb02-8c5f-48f0-a33d-fed3e9e5aab8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:25.060728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dad0fb02-8c5f-48f0-a33d-fed3e9e5aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:25.166048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 665/1327] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:25.208610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2967f4d-2ac0-4b9d-b9fc-c1283ec604a3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:25.210684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d2967f4d-2ac0-4b9d-b9fc-c1283ec604a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:25.332879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2967f4d-2ac0-4b9d-b9fc-c1283ec604a3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:25.421796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cb8bc-f014-4adf-80e9-f8a383211c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:25.422634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cb8bc-f014-4adf-80e9-f8a383211c1d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:25.422789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cb8bc-f014-4adf-80e9-f8a383211c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:25.423002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cb8bc-f014-4adf-80e9-f8a383211c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:25.424675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f5cb8bc-f014-4adf-80e9-f8a383211c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:25.443547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:25.554868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cb8bc-f014-4adf-80e9-f8a383211c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:25.556368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cb8bc-f014-4adf-80e9-f8a383211c1d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:25.578789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 663/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:27.815215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489c4f3-0f6a-4b09-b740-2df4e9c1c83e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.815893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489c4f3-0f6a-4b09-b740-2df4e9c1c83e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.815893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489c4f3-0f6a-4b09-b740-2df4e9c1c83e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.817156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 666/1329] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:27.823567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf3cb7-417d-47ef-b1fb-85afb623353a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.824065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf3cb7-417d-47ef-b1fb-85afb623353a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.824156 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf3cb7-417d-47ef-b1fb-85afb623353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.892140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f7bf3cb7-417d-47ef-b1fb-85afb623353a None None] MFA Rules not processed for user `d024a945bdaf4746aa6cddd29acc851e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:27.893396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f7bf3cb7-417d-47ef-b1fb-85afb623353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.962764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:27.997616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 664/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:28.036328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489c4f3-0f6a-4b09-b740-2df4e9c1c83e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.038764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9489c4f3-0f6a-4b09-b740-2df4e9c1c83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.150231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489c4f3-0f6a-4b09-b740-2df4e9c1c83e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.170270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:28.226119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cd628-36f6-4880-a178-21deaf5f5515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.226773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd628-36f6-4880-a178-21deaf5f5515 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.227064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd628-36f6-4880-a178-21deaf5f5515 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.227344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd628-36f6-4880-a178-21deaf5f5515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.229048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c79cd628-36f6-4880-a178-21deaf5f5515 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.355319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cd628-36f6-4880-a178-21deaf5f5515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.356808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cd628-36f6-4880-a178-21deaf5f5515 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.376966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 667/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:30.966368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce44fc-3e8d-44f7-9bc1-b3d034bcb508 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:30.966920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce44fc-3e8d-44f7-9bc1-b3d034bcb508 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:30.966920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce44fc-3e8d-44f7-9bc1-b3d034bcb508 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:30.968328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 665/1332] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:30.973920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad8a9-6b8c-41f8-a430-b2ec35cd861d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:30.974362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad8a9-6b8c-41f8-a430-b2ec35cd861d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:30.974481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad8a9-6b8c-41f8-a430-b2ec35cd861d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:31.047555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:31.050612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7dbad8a9-6b8c-41f8-a430-b2ec35cd861d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:31.052001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7dbad8a9-6b8c-41f8-a430-b2ec35cd861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:31.153954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 668/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:31.190928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce44fc-3e8d-44f7-9bc1-b3d034bcb508 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:31.192533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-82ce44fc-3e8d-44f7-9bc1-b3d034bcb508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:31.311679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce44fc-3e8d-44f7-9bc1-b3d034bcb508 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:31.395153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb9875-c152-4f4a-82b7-b2a1470d5a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:31.395800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9875-c152-4f4a-82b7-b2a1470d5a7c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:31.396084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9875-c152-4f4a-82b7-b2a1470d5a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:31.396360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9875-c152-4f4a-82b7-b2a1470d5a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:31.398744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fffb9875-c152-4f4a-82b7-b2a1470d5a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:31.426362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:31.518465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb9875-c152-4f4a-82b7-b2a1470d5a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:31.519984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb9875-c152-4f4a-82b7-b2a1470d5a7c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:31.544522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 666/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:33.696682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d412-f6ec-4b28-ba0b-116e566e83b8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:33.697277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d412-f6ec-4b28-ba0b-116e566e83b8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:33.697277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d412-f6ec-4b28-ba0b-116e566e83b8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:33.698691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 669/1335] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:33.704064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067a5e-2c99-4bb6-aca9-b1664ff895fe None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:33.704501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067a5e-2c99-4bb6-aca9-b1664ff895fe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:33.704609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067a5e-2c99-4bb6-aca9-b1664ff895fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:33.771052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-25067a5e-2c99-4bb6-aca9-b1664ff895fe None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:33.772246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-25067a5e-2c99-4bb6-aca9-b1664ff895fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:33.848751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:33.872390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 667/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:33.913026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d412-f6ec-4b28-ba0b-116e566e83b8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:33.914683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca73d412-f6ec-4b28-ba0b-116e566e83b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:34.021965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d412-f6ec-4b28-ba0b-116e566e83b8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:34.052285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:34.097877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdfdf4-630c-4336-bb6b-da287525194e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:34.098566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdfdf4-630c-4336-bb6b-da287525194e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:34.098839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdfdf4-630c-4336-bb6b-da287525194e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:34.099117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdfdf4-630c-4336-bb6b-da287525194e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:34.101548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-12fdfdf4-630c-4336-bb6b-da287525194e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:34.210201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdfdf4-630c-4336-bb6b-da287525194e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:34.211649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdfdf4-630c-4336-bb6b-da287525194e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:34.230410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 670/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:36.931309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2cbb3-c097-4281-baa7-5700b38e07b4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:36.931891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2cbb3-c097-4281-baa7-5700b38e07b4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:36.932001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2cbb3-c097-4281-baa7-5700b38e07b4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:36.933367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 668/1338] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:36.940547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed500d48-186b-4374-8097-7c6beddb2765 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:36.941079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed500d48-186b-4374-8097-7c6beddb2765 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:36.942454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed500d48-186b-4374-8097-7c6beddb2765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:37.012394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ed500d48-186b-4374-8097-7c6beddb2765 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:37.013635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed500d48-186b-4374-8097-7c6beddb2765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:37.022958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:37.139948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 671/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:37.177706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2cbb3-c097-4281-baa7-5700b38e07b4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:37.179456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-14e2cbb3-c097-4281-baa7-5700b38e07b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:37.285574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2cbb3-c097-4281-baa7-5700b38e07b4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:37.363168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae9dda-de1c-490b-a049-097553c16114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:37.363829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae9dda-de1c-490b-a049-097553c16114 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:37.364116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae9dda-de1c-490b-a049-097553c16114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:37.364387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae9dda-de1c-490b-a049-097553c16114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:37.366187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-18ae9dda-de1c-490b-a049-097553c16114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:37.412684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:37.487860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae9dda-de1c-490b-a049-097553c16114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:37.489374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae9dda-de1c-490b-a049-097553c16114 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:37.508967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 669/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:42.145614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1577bd-94b2-4d45-983a-9aa02d93ab28 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:42.146628 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1577bd-94b2-4d45-983a-9aa02d93ab28 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:42.146628 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1577bd-94b2-4d45-983a-9aa02d93ab28 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:42.148050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 672/1341] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:42.154040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c70e5e-c726-47b6-9846-6ef3f816cb29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:42.154864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c70e5e-c726-47b6-9846-6ef3f816cb29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:42.155290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c70e5e-c726-47b6-9846-6ef3f816cb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:42.239398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-71c70e5e-c726-47b6-9846-6ef3f816cb29 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:42.241910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71c70e5e-c726-47b6-9846-6ef3f816cb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:42.345910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:42.358809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 670/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:42.401250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1577bd-94b2-4d45-983a-9aa02d93ab28 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:42.402778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c1577bd-94b2-4d45-983a-9aa02d93ab28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:42.520098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1577bd-94b2-4d45-983a-9aa02d93ab28 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:42.559873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:42.599592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55cb0d-eee1-445c-bf64-a3abbcfc6cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:42.600187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55cb0d-eee1-445c-bf64-a3abbcfc6cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:42.600573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55cb0d-eee1-445c-bf64-a3abbcfc6cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:42.601006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55cb0d-eee1-445c-bf64-a3abbcfc6cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:42.602579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fe55cb0d-eee1-445c-bf64-a3abbcfc6cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:42.718981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55cb0d-eee1-445c-bf64-a3abbcfc6cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:42.720435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55cb0d-eee1-445c-bf64-a3abbcfc6cb8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:42.739356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 673/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:45.805557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fdb68-52ab-447f-8c57-9ed576551106 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:45.806050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fdb68-52ab-447f-8c57-9ed576551106 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:45.806512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fdb68-52ab-447f-8c57-9ed576551106 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:45.809112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 671/1344] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:45.814979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6868-9c6d-43ce-a30d-786015808f09 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:45.815535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6868-9c6d-43ce-a30d-786015808f09 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:45.815535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6868-9c6d-43ce-a30d-786015808f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:45.892811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2b5a6868-9c6d-43ce-a30d-786015808f09 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:45.894908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2b5a6868-9c6d-43ce-a30d-786015808f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:45.925089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:46.029530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 674/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:46.074474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fdb68-52ab-447f-8c57-9ed576551106 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:46.076041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-969fdb68-52ab-447f-8c57-9ed576551106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:46.194262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fdb68-52ab-447f-8c57-9ed576551106 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:46.272867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18cefc-a7fd-4f3a-9292-20d4edb5c470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:46.273553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18cefc-a7fd-4f3a-9292-20d4edb5c470 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:46.273865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18cefc-a7fd-4f3a-9292-20d4edb5c470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:46.274145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18cefc-a7fd-4f3a-9292-20d4edb5c470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:46.275996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d18cefc-a7fd-4f3a-9292-20d4edb5c470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:46.324352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:46.403078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18cefc-a7fd-4f3a-9292-20d4edb5c470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:46.404575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18cefc-a7fd-4f3a-9292-20d4edb5c470 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:46.429736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 672/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:50.881825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3fc35-6d3b-470f-a696-9de25aa128d5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:50.882333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3fc35-6d3b-470f-a696-9de25aa128d5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:50.882418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3fc35-6d3b-470f-a696-9de25aa128d5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:50.883992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 675/1347] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:50.889476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bacb7-42ed-4dea-8f28-de08f21e17d0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:50.889954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bacb7-42ed-4dea-8f28-de08f21e17d0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:50.890069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bacb7-42ed-4dea-8f28-de08f21e17d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:50.961801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-401bacb7-42ed-4dea-8f28-de08f21e17d0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:50.962967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-401bacb7-42ed-4dea-8f28-de08f21e17d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:51.063464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:51.066119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 673/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:51.106091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3fc35-6d3b-470f-a696-9de25aa128d5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:51.107661 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-faa3fc35-6d3b-470f-a696-9de25aa128d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:51.216715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3fc35-6d3b-470f-a696-9de25aa128d5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:51.265180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:51.291764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc196b0a-fbc3-43b2-ae93-65757ed556c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:51.292427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196b0a-fbc3-43b2-ae93-65757ed556c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:51.292712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196b0a-fbc3-43b2-ae93-65757ed556c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:51.293026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196b0a-fbc3-43b2-ae93-65757ed556c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:51.294750 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc196b0a-fbc3-43b2-ae93-65757ed556c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:51.403884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc196b0a-fbc3-43b2-ae93-65757ed556c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:51.405516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc196b0a-fbc3-43b2-ae93-65757ed556c8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:51.426332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 676/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:56.424593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faba5c-5eeb-4717-905a-7f09ced3cdd9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:56.425057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faba5c-5eeb-4717-905a-7f09ced3cdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:56.425523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faba5c-5eeb-4717-905a-7f09ced3cdd9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:56.429785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 674/1350] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:56.436844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238e3-d9fb-480c-b25c-1e7152bd0402 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:56.436844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238e3-d9fb-480c-b25c-1e7152bd0402 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:56.436844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238e3-d9fb-480c-b25c-1e7152bd0402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:56.534916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f08238e3-d9fb-480c-b25c-1e7152bd0402 None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:56.537068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f08238e3-d9fb-480c-b25c-1e7152bd0402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:56.562284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:56.724812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 677/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:37:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:38:01.174876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da75e7-7888-4afb-b368-a1e68f383060 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:01.175441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da75e7-7888-4afb-b368-a1e68f383060 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:01.175615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da75e7-7888-4afb-b368-a1e68f383060 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:01.177778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 675/1352] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:01.184172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c1663-d8eb-4c48-a1ff-938b70bd17f1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:01.184743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c1663-d8eb-4c48-a1ff-938b70bd17f1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:01.184743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c1663-d8eb-4c48-a1ff-938b70bd17f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:01.275136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-474c1663-d8eb-4c48-a1ff-938b70bd17f1 None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:01.276612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-474c1663-d8eb-4c48-a1ff-938b70bd17f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:01.360679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:01.409649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 678/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:38:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:02.664969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653f27-9d56-41c7-ae02-177565c12510 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:02.665494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653f27-9d56-41c7-ae02-177565c12510 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:02.665494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653f27-9d56-41c7-ae02-177565c12510 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:02.666798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 676/1354] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:02.671934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14a7f3-aeb3-4643-9b58-bc5bb83d7eff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:02.672345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14a7f3-aeb3-4643-9b58-bc5bb83d7eff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:02.672501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14a7f3-aeb3-4643-9b58-bc5bb83d7eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:02.751998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4e14a7f3-aeb3-4643-9b58-bc5bb83d7eff None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:02.753253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4e14a7f3-aeb3-4643-9b58-bc5bb83d7eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:02.865301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 679/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:38:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:06.367258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309dde-6743-4a05-a709-ea00e3f32962 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:06.368280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309dde-6743-4a05-a709-ea00e3f32962 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:06.368638 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309dde-6743-4a05-a709-ea00e3f32962 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:06.370576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 677/1356] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:06.377973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76921508-ed6d-4485-9801-af4744b28c6b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:06.378537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76921508-ed6d-4485-9801-af4744b28c6b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:06.378537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76921508-ed6d-4485-9801-af4744b28c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:06.439710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:06.496914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-76921508-ed6d-4485-9801-af4744b28c6b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:06.498560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-76921508-ed6d-4485-9801-af4744b28c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:06.624715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 680/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:06.685381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15b070-1c41-4295-aff4-da47c1c1541c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:06.685942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15b070-1c41-4295-aff4-da47c1c1541c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:06.686363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15b070-1c41-4295-aff4-da47c1c1541c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:06.688166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 678/1358] 10.222.0.49 () {54 vars in 962 bytes} [Mon Jul 18 10:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:06.697876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00b3da-6156-479c-ae21-a9f2e873cea9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:06.697876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00b3da-6156-479c-ae21-a9f2e873cea9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:06.697876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00b3da-6156-479c-ae21-a9f2e873cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:06.806840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1f00b3da-6156-479c-ae21-a9f2e873cea9 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:06.808132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1f00b3da-6156-479c-ae21-a9f2e873cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:06.891015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:06.987716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 681/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Jul 18 10:38:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:06.996848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15b070-1c41-4295-aff4-da47c1c1541c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:06.998455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ae15b070-1c41-4295-aff4-da47c1c1541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:07.176384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15b070-1c41-4295-aff4-da47c1c1541c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:07.273487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:07.324531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b52b52-7c81-406a-b317-4237349bc757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:07.325423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52b52-7c81-406a-b317-4237349bc757 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:07.325768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52b52-7c81-406a-b317-4237349bc757 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:07.326125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52b52-7c81-406a-b317-4237349bc757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:07.327843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a9b52b52-7c81-406a-b317-4237349bc757 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:07.444507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b52b52-7c81-406a-b317-4237349bc757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:07.446201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b52b52-7c81-406a-b317-4237349bc757 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:07.470662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 679/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:38:09.767782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c40ec-6128-4ae6-bc9c-7f3c8f1d3ff3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:09.768314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c40ec-6128-4ae6-bc9c-7f3c8f1d3ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:09.768314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c40ec-6128-4ae6-bc9c-7f3c8f1d3ff3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:09.773473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 682/1361] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:38:09.778051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa9b6f-e8ab-4a49-99f8-eeaa36b5f2fd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:09.778751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa9b6f-e8ab-4a49-99f8-eeaa36b5f2fd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:09.778751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa9b6f-e8ab-4a49-99f8-eeaa36b5f2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:09.884398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-18aa9b6f-e8ab-4a49-99f8-eeaa36b5f2fd None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:09.886770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-18aa9b6f-e8ab-4a49-99f8-eeaa36b5f2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:09.951145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:10.015897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 680/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:10.065488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c40ec-6128-4ae6-bc9c-7f3c8f1d3ff3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:10.067012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f8c40ec-6128-4ae6-bc9c-7f3c8f1d3ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:10.297970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c40ec-6128-4ae6-bc9c-7f3c8f1d3ff3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:10.311083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:10.443341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41ca2e-b3d1-42ea-b83e-2e8165ead8e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:10.444013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ca2e-b3d1-42ea-b83e-2e8165ead8e7 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:10.444300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ca2e-b3d1-42ea-b83e-2e8165ead8e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:10.444576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ca2e-b3d1-42ea-b83e-2e8165ead8e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:10.446400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9d41ca2e-b3d1-42ea-b83e-2e8165ead8e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:10.557310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41ca2e-b3d1-42ea-b83e-2e8165ead8e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:10.559241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41ca2e-b3d1-42ea-b83e-2e8165ead8e7 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:10.589441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 683/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:13.245379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3efde9-3e5e-4990-a4af-aaf0cfd3866b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:13.245901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3efde9-3e5e-4990-a4af-aaf0cfd3866b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:13.245901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3efde9-3e5e-4990-a4af-aaf0cfd3866b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:13.247287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 681/1364] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:13.252945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a9ccc-6bba-4609-95b1-3c094f1c1e25 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:13.253390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a9ccc-6bba-4609-95b1-3c094f1c1e25 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:13.253743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a9ccc-6bba-4609-95b1-3c094f1c1e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:13.273704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:13.332188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c67a9ccc-6bba-4609-95b1-3c094f1c1e25 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:13.333964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c67a9ccc-6bba-4609-95b1-3c094f1c1e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:13.456834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 684/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:13.499362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace97b8-f085-4008-a5aa-8f25c5a3a1b6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:13.499650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace97b8-f085-4008-a5aa-8f25c5a3a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:13.500118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace97b8-f085-4008-a5aa-8f25c5a3a1b6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:13.501519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 682/1366] 10.222.0.49 () {54 vars in 962 bytes} [Mon Jul 18 10:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:13.510856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4cd18-ff2a-4762-a2f3-eea38d94de52 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:13.511432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4cd18-ff2a-4762-a2f3-eea38d94de52 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:13.511921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4cd18-ff2a-4762-a2f3-eea38d94de52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:13.611007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d6b4cd18-ff2a-4762-a2f3-eea38d94de52 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:13.612459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6b4cd18-ff2a-4762-a2f3-eea38d94de52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:13.655404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:13.753329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 685/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Jul 18 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:13.761785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace97b8-f085-4008-a5aa-8f25c5a3a1b6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:13.763289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ace97b8-f085-4008-a5aa-8f25c5a3a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:13.951138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace97b8-f085-4008-a5aa-8f25c5a3a1b6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:13.975064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:14.074516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216ff14-a8f8-46b1-85f8-b54f980eb5c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:14.075077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216ff14-a8f8-46b1-85f8-b54f980eb5c1 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:14.075374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216ff14-a8f8-46b1-85f8-b54f980eb5c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:14.075624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216ff14-a8f8-46b1-85f8-b54f980eb5c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:14.077313 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c216ff14-a8f8-46b1-85f8-b54f980eb5c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:14.176692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216ff14-a8f8-46b1-85f8-b54f980eb5c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:14.178490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216ff14-a8f8-46b1-85f8-b54f980eb5c1 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:14.200826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 683/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:15.034421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba592f8-8b70-4335-b78a-2c7cfe4a107d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:15.035033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba592f8-8b70-4335-b78a-2c7cfe4a107d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:15.035033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba592f8-8b70-4335-b78a-2c7cfe4a107d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:15.036587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 686/1369] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:38:15.042118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864af976-abaf-4676-8904-2c5ea1422db9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:15.042579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864af976-abaf-4676-8904-2c5ea1422db9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:15.042670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864af976-abaf-4676-8904-2c5ea1422db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:15.076858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:15.120226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-864af976-abaf-4676-8904-2c5ea1422db9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:15.123044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-864af976-abaf-4676-8904-2c5ea1422db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:15.246975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 684/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:15.629210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c2e88-4496-419e-81ab-e89870c6fb41 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:15.629210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c2e88-4496-419e-81ab-e89870c6fb41 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:15.629210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c2e88-4496-419e-81ab-e89870c6fb41 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:15.630784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 687/1371] 10.222.0.49 () {54 vars in 964 bytes} [Mon Jul 18 10:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:15.640143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae63ed4-7991-4b7f-8b26-bd0a6d74a679 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:15.640143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae63ed4-7991-4b7f-8b26-bd0a6d74a679 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:15.640143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae63ed4-7991-4b7f-8b26-bd0a6d74a679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:15.753826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2ae63ed4-7991-4b7f-8b26-bd0a6d74a679 None None] MFA Rules not processed for user `62e462a1b125428f9893459d5d965ced`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:15.755437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ae63ed4-7991-4b7f-8b26-bd0a6d74a679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:15.820089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:15.856235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 685/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Jul 18 10:38:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:15.864632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c2e88-4496-419e-81ab-e89870c6fb41 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:15.866167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-617c2e88-4496-419e-81ab-e89870c6fb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:15.995785 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c2e88-4496-419e-81ab-e89870c6fb41 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:16.050656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:16.091044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b46c2-37ff-4338-859a-e964321b90b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:16.091738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b46c2-37ff-4338-859a-e964321b90b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:16.095491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b46c2-37ff-4338-859a-e964321b90b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:16.095491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b46c2-37ff-4338-859a-e964321b90b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:16.095491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c18b46c2-37ff-4338-859a-e964321b90b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:16.227381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b46c2-37ff-4338-859a-e964321b90b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:16.228913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b46c2-37ff-4338-859a-e964321b90b5 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:16.257878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 688/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:16.554088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8459168-a0a8-40e9-9b2a-c4106049453c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:16.554088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8459168-a0a8-40e9-9b2a-c4106049453c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:16.555075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8459168-a0a8-40e9-9b2a-c4106049453c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:16.557170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 686/1374] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:16.561652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3527-ff77-42c8-8c9d-accb65441549 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:16.562024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3527-ff77-42c8-8c9d-accb65441549 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:16.562397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3527-ff77-42c8-8c9d-accb65441549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:16.636548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-602f3527-ff77-42c8-8c9d-accb65441549 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:16.637833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-602f3527-ff77-42c8-8c9d-accb65441549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:16.659831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:16.740462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 689/1375] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:38:16.785576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8459168-a0a8-40e9-9b2a-c4106049453c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:16.787160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8459168-a0a8-40e9-9b2a-c4106049453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:16.956656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8459168-a0a8-40e9-9b2a-c4106049453c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:17.015472 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:17.083234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4086f-5134-433c-b9be-8fa8ed7086ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:17.083905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4086f-5134-433c-b9be-8fa8ed7086ce service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:17.084190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4086f-5134-433c-b9be-8fa8ed7086ce service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:17.084459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4086f-5134-433c-b9be-8fa8ed7086ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:17.086291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fa4086f-5134-433c-b9be-8fa8ed7086ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:17.185968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4086f-5134-433c-b9be-8fa8ed7086ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:17.187580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4086f-5134-433c-b9be-8fa8ed7086ce service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:17.206216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 687/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:19.460028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45edc7-c0fe-4934-9cb5-8d10cd2002b0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:19.460878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45edc7-c0fe-4934-9cb5-8d10cd2002b0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:19.460878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45edc7-c0fe-4934-9cb5-8d10cd2002b0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:19.462165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 690/1377] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:19.468562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b23122-5968-48f6-a192-cc72094d8548 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:19.469327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b23122-5968-48f6-a192-cc72094d8548 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:19.469327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b23122-5968-48f6-a192-cc72094d8548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:19.545608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d5b23122-5968-48f6-a192-cc72094d8548 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:19.547132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5b23122-5968-48f6-a192-cc72094d8548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:19.555028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:19.684991 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 688/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:38:19.726639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45edc7-c0fe-4934-9cb5-8d10cd2002b0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:19.728249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2e45edc7-c0fe-4934-9cb5-8d10cd2002b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:19.891050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45edc7-c0fe-4934-9cb5-8d10cd2002b0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:19.895126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:20.019688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08d57e-b2c9-4ca0-a3fd-3a6ccf7c34a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:20.021122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08d57e-b2c9-4ca0-a3fd-3a6ccf7c34a5 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:20.021437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08d57e-b2c9-4ca0-a3fd-3a6ccf7c34a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:20.021715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08d57e-b2c9-4ca0-a3fd-3a6ccf7c34a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:20.023427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d08d57e-b2c9-4ca0-a3fd-3a6ccf7c34a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:20.163467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08d57e-b2c9-4ca0-a3fd-3a6ccf7c34a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:20.165044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08d57e-b2c9-4ca0-a3fd-3a6ccf7c34a5 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:20.192869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 691/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:22.499882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b061ab-67b7-418d-93ee-74f3d1c293fd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:22.500457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b061ab-67b7-418d-93ee-74f3d1c293fd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:22.500457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b061ab-67b7-418d-93ee-74f3d1c293fd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:22.501996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 689/1380] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:22.507897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858fdf-2561-47b8-8f77-b3cda5aa73f6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:22.508498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858fdf-2561-47b8-8f77-b3cda5aa73f6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:22.508933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858fdf-2561-47b8-8f77-b3cda5aa73f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:22.521014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:22.591107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-25858fdf-2561-47b8-8f77-b3cda5aa73f6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:22.592223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25858fdf-2561-47b8-8f77-b3cda5aa73f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:22.729082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 692/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:22.771475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b061ab-67b7-418d-93ee-74f3d1c293fd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:22.774310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50b061ab-67b7-418d-93ee-74f3d1c293fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:22.938828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:22.960260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b061ab-67b7-418d-93ee-74f3d1c293fd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:23.121385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54bf1b-38d3-4f4c-b72b-1dd9fd54ee6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:23.121971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54bf1b-38d3-4f4c-b72b-1dd9fd54ee6d service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:23.122255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54bf1b-38d3-4f4c-b72b-1dd9fd54ee6d service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:23.122532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54bf1b-38d3-4f4c-b72b-1dd9fd54ee6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:23.124250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f54bf1b-38d3-4f4c-b72b-1dd9fd54ee6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:23.241112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54bf1b-38d3-4f4c-b72b-1dd9fd54ee6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:23.242822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54bf1b-38d3-4f4c-b72b-1dd9fd54ee6d service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:23.256513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:23.266022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 690/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:25.876619 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e791-a01a-4264-ad03-8f2ef89e418e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:25.877258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e791-a01a-4264-ad03-8f2ef89e418e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:25.877258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e791-a01a-4264-ad03-8f2ef89e418e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:25.878587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 693/1383] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:25.885168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cae7a7-58f8-40cf-8481-3539a216a441 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:25.885827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cae7a7-58f8-40cf-8481-3539a216a441 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:25.885827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cae7a7-58f8-40cf-8481-3539a216a441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:25.977267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-33cae7a7-58f8-40cf-8481-3539a216a441 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:25.978511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-33cae7a7-58f8-40cf-8481-3539a216a441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:26.086491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 691/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:26.128751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e791-a01a-4264-ad03-8f2ef89e418e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:26.131021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc55e791-a01a-4264-ad03-8f2ef89e418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:26.253387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:26.333718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e791-a01a-4264-ad03-8f2ef89e418e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:26.464186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e9e4-9c2d-4547-b03c-5279338bedd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:26.464877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e9e4-9c2d-4547-b03c-5279338bedd9 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:26.465162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e9e4-9c2d-4547-b03c-5279338bedd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:26.465444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e9e4-9c2d-4547-b03c-5279338bedd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:26.467153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2043e9e4-9c2d-4547-b03c-5279338bedd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:26.524169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:26.568715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e9e4-9c2d-4547-b03c-5279338bedd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:26.570432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e9e4-9c2d-4547-b03c-5279338bedd9 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:26.591238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 694/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:28.653825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a38095-df42-4185-9814-5f7577ecee34 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:28.654363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a38095-df42-4185-9814-5f7577ecee34 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:28.654456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a38095-df42-4185-9814-5f7577ecee34 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:28.655769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 692/1386] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:28.661272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7a03d-d9ea-421c-93ca-d295e45a5f77 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:28.661826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7a03d-d9ea-421c-93ca-d295e45a5f77 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:28.661826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7a03d-d9ea-421c-93ca-d295e45a5f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:28.739286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3bc7a03d-d9ea-421c-93ca-d295e45a5f77 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:28.740669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3bc7a03d-d9ea-421c-93ca-d295e45a5f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:28.880046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 695/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:28.928447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a38095-df42-4185-9814-5f7577ecee34 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:28.930023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66a38095-df42-4185-9814-5f7577ecee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:29.016231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:29.108572 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a38095-df42-4185-9814-5f7577ecee34 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:29.261011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0208a-8255-4907-96bf-341fa4a8b34c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:29.261726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0208a-8255-4907-96bf-341fa4a8b34c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:29.262070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0208a-8255-4907-96bf-341fa4a8b34c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:29.262351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0208a-8255-4907-96bf-341fa4a8b34c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:29.264066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c1e0208a-8255-4907-96bf-341fa4a8b34c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:29.309158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:29.371100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0208a-8255-4907-96bf-341fa4a8b34c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:29.372921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0208a-8255-4907-96bf-341fa4a8b34c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:29.396194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 693/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:38:31.674980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f312-7923-4d9d-b538-50560c561799 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:31.675547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f312-7923-4d9d-b538-50560c561799 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:31.675547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f312-7923-4d9d-b538-50560c561799 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:31.677753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 696/1389] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:31.682957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821bd06-65ca-43fd-8b56-858547cf811d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:31.683386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821bd06-65ca-43fd-8b56-858547cf811d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:31.683504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821bd06-65ca-43fd-8b56-858547cf811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:31.760201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f821bd06-65ca-43fd-8b56-858547cf811d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:31.761419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f821bd06-65ca-43fd-8b56-858547cf811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:31.879915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 694/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:38:31.925853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071f312-7923-4d9d-b538-50560c561799 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:31.927498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a071f312-7923-4d9d-b538-50560c561799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:31.962718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:32.086351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071f312-7923-4d9d-b538-50560c561799 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:32.193374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:32.205328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ca2f5-9265-4e33-9c5d-d4e81d8052cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:32.205992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ca2f5-9265-4e33-9c5d-d4e81d8052cc service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:32.206278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ca2f5-9265-4e33-9c5d-d4e81d8052cc service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:32.206552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ca2f5-9265-4e33-9c5d-d4e81d8052cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:32.209328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4d4ca2f5-9265-4e33-9c5d-d4e81d8052cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:32.302354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ca2f5-9265-4e33-9c5d-d4e81d8052cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:32.303971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ca2f5-9265-4e33-9c5d-d4e81d8052cc service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:32.323293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 697/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:38:32.405827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86f421-5d06-46b9-a493-9d5b35f294a0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:32.406118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86f421-5d06-46b9-a493-9d5b35f294a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:32.406401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86f421-5d06-46b9-a493-9d5b35f294a0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:32.407820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 695/1392] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:32.413616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cb7b2-7032-4eb6-afe8-560ba6e5f5ea None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:32.413953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cb7b2-7032-4eb6-afe8-560ba6e5f5ea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:32.414340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cb7b2-7032-4eb6-afe8-560ba6e5f5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:32.496027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b54cb7b2-7032-4eb6-afe8-560ba6e5f5ea None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:32.497262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b54cb7b2-7032-4eb6-afe8-560ba6e5f5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:32.557932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:32.598843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 698/1393] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:32.655503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e95e0-65dd-4a31-87c4-79e3cc25d069 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:32.655797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e95e0-65dd-4a31-87c4-79e3cc25d069 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:32.656140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e95e0-65dd-4a31-87c4-79e3cc25d069 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:32.657613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 696/1394] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 18 10:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:32.663486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edd32a-b5c8-47d6-925f-7e7b20cf9046 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:32.663782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edd32a-b5c8-47d6-925f-7e7b20cf9046 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:32.664061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edd32a-b5c8-47d6-925f-7e7b20cf9046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:32.739820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b4edd32a-b5c8-47d6-925f-7e7b20cf9046 None None] MFA Rules not processed for user `80535e7da62e4790931fd74000653411`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:32.741066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b4edd32a-b5c8-47d6-925f-7e7b20cf9046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:32.828867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 699/1395] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 18 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:32.837499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e95e0-65dd-4a31-87c4-79e3cc25d069 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:32.839211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-245e95e0-65dd-4a31-87c4-79e3cc25d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:32.854873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:32.953825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e95e0-65dd-4a31-87c4-79e3cc25d069 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:33.030116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c8142-1824-42c6-86a6-1573d8e1f23c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:33.030768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c8142-1824-42c6-86a6-1573d8e1f23c service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:33.031053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c8142-1824-42c6-86a6-1573d8e1f23c service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:33.031330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c8142-1824-42c6-86a6-1573d8e1f23c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:33.033008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d28c8142-1824-42c6-86a6-1573d8e1f23c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:33.108397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:33.150663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c8142-1824-42c6-86a6-1573d8e1f23c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:33.152151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c8142-1824-42c6-86a6-1573d8e1f23c service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:33.170640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 697/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:34.083508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce4f4f-0dda-46ff-a337-281d65153535 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:34.083508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce4f4f-0dda-46ff-a337-281d65153535 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:34.083508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce4f4f-0dda-46ff-a337-281d65153535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:34.155868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d5ce4f4f-0dda-46ff-a337-281d65153535 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:34.157130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d5ce4f4f-0dda-46ff-a337-281d65153535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:34.166913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:34.248450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 700/1397] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:38:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:34.293250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792edc7-c2af-46ad-830f-6a992daf8cf3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:34.294178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792edc7-c2af-46ad-830f-6a992daf8cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:34.294459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792edc7-c2af-46ad-830f-6a992daf8cf3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:34.296173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 698/1398] 10.222.0.49 () {54 vars in 943 bytes} [Mon Jul 18 10:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:34.303140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f982fd7-16c1-49b4-b5df-c22026bbce41 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:34.303507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f982fd7-16c1-49b4-b5df-c22026bbce41 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:34.303790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f982fd7-16c1-49b4-b5df-c22026bbce41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:34.384858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2f982fd7-16c1-49b4-b5df-c22026bbce41 None None] MFA Rules not processed for user `642a42337be14f10b410903df57f7527`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:34.386489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f982fd7-16c1-49b4-b5df-c22026bbce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:34.481101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:34.506302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 701/1399] 10.222.0.49 () {58 vars in 1040 bytes} [Mon Jul 18 10:38:34 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:34.516111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792edc7-c2af-46ad-830f-6a992daf8cf3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:34.517657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f792edc7-c2af-46ad-830f-6a992daf8cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:34.671375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792edc7-c2af-46ad-830f-6a992daf8cf3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:34.712005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:34.779186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44274-6962-47a9-a385-02cfa288a88e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:34.779585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44274-6962-47a9-a385-02cfa288a88e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:34.779902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44274-6962-47a9-a385-02cfa288a88e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:34.781454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 702/1400] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:38:34.786632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:34.787376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:34.787668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:34.787943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:34.788048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00a9e-7305-4a2c-8fb8-961e50459446 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:34.788387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00a9e-7305-4a2c-8fb8-961e50459446 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:34.788795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00a9e-7305-4a2c-8fb8-961e50459446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:34.789719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:34.865962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-17f00a9e-7305-4a2c-8fb8-961e50459446 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:34.867199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-17f00a9e-7305-4a2c-8fb8-961e50459446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:34.883480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:34.885082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:34.893640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 699/1401] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:38:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:38:34.994503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 703/1402] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:38:35.035968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:35.037229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:35.105707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:35.133805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313da32-b62d-4048-8945-4d06a2a41352 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:35.262413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d63593-d794-48d0-bde8-6e156ad34cd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:35.262987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63593-d794-48d0-bde8-6e156ad34cd3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:35.263270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63593-d794-48d0-bde8-6e156ad34cd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:35.263547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63593-d794-48d0-bde8-6e156ad34cd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:35.265314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72d63593-d794-48d0-bde8-6e156ad34cd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:35.359859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:35.363824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d63593-d794-48d0-bde8-6e156ad34cd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:35.365804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d63593-d794-48d0-bde8-6e156ad34cd3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:35.386447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 700/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:37.666310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63274515-8805-45a1-bbe7-d5b4aa9ae803 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:37.666874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63274515-8805-45a1-bbe7-d5b4aa9ae803 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:37.666874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63274515-8805-45a1-bbe7-d5b4aa9ae803 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:37.668251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 704/1404] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:37.673643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d684-aa24-4912-a743-84d95b1b3004 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:37.674125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d684-aa24-4912-a743-84d95b1b3004 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:37.674225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d684-aa24-4912-a743-84d95b1b3004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:37.750650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4e5d684-aa24-4912-a743-84d95b1b3004 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:37.751889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4e5d684-aa24-4912-a743-84d95b1b3004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:37.875293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 701/1405] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:38:37.925811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63274515-8805-45a1-bbe7-d5b4aa9ae803 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:37.927714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63274515-8805-45a1-bbe7-d5b4aa9ae803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:38.005098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:38.108161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63274515-8805-45a1-bbe7-d5b4aa9ae803 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:38.251656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c30ca6-4ee2-4a96-862e-3495f9cb1787 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:38.252438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c30ca6-4ee2-4a96-862e-3495f9cb1787 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:38.252727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c30ca6-4ee2-4a96-862e-3495f9cb1787 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:38.253056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c30ca6-4ee2-4a96-862e-3495f9cb1787 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:38.255485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-13c30ca6-4ee2-4a96-862e-3495f9cb1787 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:38.282126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:38.365977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c30ca6-4ee2-4a96-862e-3495f9cb1787 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:38.367469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c30ca6-4ee2-4a96-862e-3495f9cb1787 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:38.393123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 705/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:41.885930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c3d74-01fc-467a-977d-c74fc54a5104 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:41.887096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c3d74-01fc-467a-977d-c74fc54a5104 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:41.887096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c3d74-01fc-467a-977d-c74fc54a5104 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:41.889080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 702/1407] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:38:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:38:41.898715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b76e-e46b-4862-98e2-dda6407deda7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:41.898715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b76e-e46b-4862-98e2-dda6407deda7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:41.898715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b76e-e46b-4862-98e2-dda6407deda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:41.975140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0896b76e-e46b-4862-98e2-dda6407deda7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:41.976692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0896b76e-e46b-4862-98e2-dda6407deda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:42.102857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:42.108472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 706/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:42.154956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c3d74-01fc-467a-977d-c74fc54a5104 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:42.156526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-617c3d74-01fc-467a-977d-c74fc54a5104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:42.241072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:42.336182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c3d74-01fc-467a-977d-c74fc54a5104 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:42.486466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba91ce-fd2d-4df5-8bb6-a34bc3c65734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:42.487337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba91ce-fd2d-4df5-8bb6-a34bc3c65734 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:42.487619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba91ce-fd2d-4df5-8bb6-a34bc3c65734 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:42.488556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba91ce-fd2d-4df5-8bb6-a34bc3c65734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:42.490712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0ba91ce-fd2d-4df5-8bb6-a34bc3c65734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:42.523653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:42.603159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba91ce-fd2d-4df5-8bb6-a34bc3c65734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:42.607432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba91ce-fd2d-4df5-8bb6-a34bc3c65734 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:42.627388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 703/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:52.624937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=34 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:59.656535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537021e-27ed-41c4-b0cc-edc46a22844d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:59.657172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537021e-27ed-41c4-b0cc-edc46a22844d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:59.657172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537021e-27ed-41c4-b0cc-edc46a22844d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:59.658540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 707/1410] 10.222.0.49 () {54 vars in 918 bytes} [Mon Jul 18 10:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:59.664407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea7e1-bdfc-48cd-b4fd-1ea09905a492 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:59.664918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea7e1-bdfc-48cd-b4fd-1ea09905a492 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:59.665014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea7e1-bdfc-48cd-b4fd-1ea09905a492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:59.750069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-454ea7e1-bdfc-48cd-b4fd-1ea09905a492 None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:59.751261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-454ea7e1-bdfc-48cd-b4fd-1ea09905a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:59.872376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 704/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Jul 18 10:38:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:59.888841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dc6c6-230a-40de-891f-11daac49ea76 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:59.889138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dc6c6-230a-40de-891f-11daac49ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:59.889420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dc6c6-230a-40de-891f-11daac49ea76 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:59.890831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 708/1412] 10.222.0.49 () {54 vars in 918 bytes} [Mon Jul 18 10:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:38:59.896441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5ea7-e275-4334-9f9f-a1b16490b562 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:59.896738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5ea7-e275-4334-9f9f-a1b16490b562 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:59.897069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5ea7-e275-4334-9f9f-a1b16490b562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:59.980556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:59.984263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-989b5ea7-e275-4334-9f9f-a1b16490b562 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:59.985779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-989b5ea7-e275-4334-9f9f-a1b16490b562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:00.109079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 705/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Jul 18 10:38:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:02.645397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26972487-5421-4f6f-b94e-0f76e09f96a9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:02.645974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26972487-5421-4f6f-b94e-0f76e09f96a9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:02.645974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26972487-5421-4f6f-b94e-0f76e09f96a9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:02.647334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:02.653107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4d09b-25ee-4225-abc0-a966a52466c0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:02.653602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4d09b-25ee-4225-abc0-a966a52466c0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:02.653705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4d09b-25ee-4225-abc0-a966a52466c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:02.722526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c6d4d09b-25ee-4225-abc0-a966a52466c0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:02.723774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6d4d09b-25ee-4225-abc0-a966a52466c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:02.780157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:02.827167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:02.871598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26972487-5421-4f6f-b94e-0f76e09f96a9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:02.873154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-26972487-5421-4f6f-b94e-0f76e09f96a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:03.029537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26972487-5421-4f6f-b94e-0f76e09f96a9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:03.104024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:03.144310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea14d9-9c2f-43e1-bf96-81dd6b146329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:03.145016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea14d9-9c2f-43e1-bf96-81dd6b146329 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:03.145275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea14d9-9c2f-43e1-bf96-81dd6b146329 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:03.145552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea14d9-9c2f-43e1-bf96-81dd6b146329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:03.147206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b9ea14d9-9c2f-43e1-bf96-81dd6b146329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:03.241845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea14d9-9c2f-43e1-bf96-81dd6b146329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:03.243486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea14d9-9c2f-43e1-bf96-81dd6b146329 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:03.262010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 710/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:04.185136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2ccba-4114-4c0b-91b8-c3c6e444052e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:04.185784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2ccba-4114-4c0b-91b8-c3c6e444052e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:04.185784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2ccba-4114-4c0b-91b8-c3c6e444052e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:04.187075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:39:04.193174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69079d3e-b071-4f1c-8fc9-5c99ad5f6324 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:04.193664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69079d3e-b071-4f1c-8fc9-5c99ad5f6324 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:04.193811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69079d3e-b071-4f1c-8fc9-5c99ad5f6324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:04.272048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-69079d3e-b071-4f1c-8fc9-5c99ad5f6324 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:04.273401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69079d3e-b071-4f1c-8fc9-5c99ad5f6324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:04.303963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:04.376236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:04.533853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2ccba-4114-4c0b-91b8-c3c6e444052e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:04.535975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4a2ccba-4114-4c0b-91b8-c3c6e444052e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:04.690386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2ccba-4114-4c0b-91b8-c3c6e444052e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:04.734143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:04.809586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6148c-03b4-4b22-a161-f1426a27392f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:04.810245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6148c-03b4-4b22-a161-f1426a27392f service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:04.810609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6148c-03b4-4b22-a161-f1426a27392f service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:04.810814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6148c-03b4-4b22-a161-f1426a27392f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:04.812477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ac6148c-03b4-4b22-a161-f1426a27392f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:04.921580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6148c-03b4-4b22-a161-f1426a27392f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:04.923247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6148c-03b4-4b22-a161-f1426a27392f service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:04.949363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 708/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:05.869768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e305b6d-94e0-4386-8873-058942188358 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:05.870378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e305b6d-94e0-4386-8873-058942188358 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:05.870378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e305b6d-94e0-4386-8873-058942188358 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:05.871703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:05.876961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebf03-d05f-456e-9982-c9fc7d7fcfb2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:05.877427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebf03-d05f-456e-9982-c9fc7d7fcfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:05.877558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebf03-d05f-456e-9982-c9fc7d7fcfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:05.936063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:05.960851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-809ebf03-d05f-456e-9982-c9fc7d7fcfb2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:05.962117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-809ebf03-d05f-456e-9982-c9fc7d7fcfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:06.064091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:06.219751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e305b6d-94e0-4386-8873-058942188358 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:06.221389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9e305b6d-94e0-4386-8873-058942188358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:06.380012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e305b6d-94e0-4386-8873-058942188358 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:06.392725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:06.508380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184f393-a666-474a-be00-a31591f2cdb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:06.509072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184f393-a666-474a-be00-a31591f2cdb7 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:06.509365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184f393-a666-474a-be00-a31591f2cdb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:06.509653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184f393-a666-474a-be00-a31591f2cdb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:06.511365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c184f393-a666-474a-be00-a31591f2cdb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:06.610316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184f393-a666-474a-be00-a31591f2cdb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:06.611938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184f393-a666-474a-be00-a31591f2cdb7 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:06.632604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 713/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:07.515923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492f0ce-4464-45aa-bef5-3052f9ba97cc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:07.516480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492f0ce-4464-45aa-bef5-3052f9ba97cc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:07.516480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492f0ce-4464-45aa-bef5-3052f9ba97cc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:07.517909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:07.523632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169bd59-795f-4fc7-9930-a4fdbf7d70aa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:07.524192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169bd59-795f-4fc7-9930-a4fdbf7d70aa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:07.524192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169bd59-795f-4fc7-9930-a4fdbf7d70aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:07.545948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:07.611914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6169bd59-795f-4fc7-9930-a4fdbf7d70aa None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:07.613773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6169bd59-795f-4fc7-9930-a4fdbf7d70aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:07.720133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:07.876580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492f0ce-4464-45aa-bef5-3052f9ba97cc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:07.878151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8492f0ce-4464-45aa-bef5-3052f9ba97cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:08.001483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:08.044375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492f0ce-4464-45aa-bef5-3052f9ba97cc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:08.171363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d95131-a2b6-4650-9d11-1b855dc04488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:08.171923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95131-a2b6-4650-9d11-1b855dc04488 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:08.172208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95131-a2b6-4650-9d11-1b855dc04488 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:08.172486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95131-a2b6-4650-9d11-1b855dc04488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:08.174191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a1d95131-a2b6-4650-9d11-1b855dc04488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:08.256917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:08.272354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d95131-a2b6-4650-9d11-1b855dc04488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:08.274077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d95131-a2b6-4650-9d11-1b855dc04488 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:08.295116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 711/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:09.246539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904035b-cbda-446d-ab08-773f08904a34 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:09.247255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904035b-cbda-446d-ab08-773f08904a34 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:09.247255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904035b-cbda-446d-ab08-773f08904a34 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:09.248747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:39:09.256921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e13ee-5cd6-428d-b6ec-e95639247145 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:09.257582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e13ee-5cd6-428d-b6ec-e95639247145 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:09.257907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e13ee-5cd6-428d-b6ec-e95639247145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:09.362550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-750e13ee-5cd6-428d-b6ec-e95639247145 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:09.364505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-750e13ee-5cd6-428d-b6ec-e95639247145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:09.501243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:09.667477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904035b-cbda-446d-ab08-773f08904a34 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:09.670096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5904035b-cbda-446d-ab08-773f08904a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:09.828559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:09.909846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904035b-cbda-446d-ab08-773f08904a34 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:10.056310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5914660-2d2b-42de-ad87-ed082aa33ca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:10.057045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914660-2d2b-42de-ad87-ed082aa33ca5 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:10.057384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914660-2d2b-42de-ad87-ed082aa33ca5 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:10.057716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914660-2d2b-42de-ad87-ed082aa33ca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:10.059683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f5914660-2d2b-42de-ad87-ed082aa33ca5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:10.121350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:10.191874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5914660-2d2b-42de-ad87-ed082aa33ca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:10.194714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5914660-2d2b-42de-ad87-ed082aa33ca5 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:10.228598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 716/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:11.265618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039766f5-2138-4adf-8fed-e8dc10fe1d08 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:11.266332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039766f5-2138-4adf-8fed-e8dc10fe1d08 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:11.266332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039766f5-2138-4adf-8fed-e8dc10fe1d08 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:11.268289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:11.275435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245e71-942b-421c-b02e-161b7d715d13 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:11.276064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245e71-942b-421c-b02e-161b7d715d13 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:11.276170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245e71-942b-421c-b02e-161b7d715d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:11.363935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d1245e71-942b-421c-b02e-161b7d715d13 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:11.365157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d1245e71-942b-421c-b02e-161b7d715d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:11.478982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:11.703469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039766f5-2138-4adf-8fed-e8dc10fe1d08 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:11.705112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-039766f5-2138-4adf-8fed-e8dc10fe1d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:11.763411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:11.892577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039766f5-2138-4adf-8fed-e8dc10fe1d08 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:12.025050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329c287-84c0-49eb-a78f-85f6f2147336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:12.025889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329c287-84c0-49eb-a78f-85f6f2147336 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:12.026296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329c287-84c0-49eb-a78f-85f6f2147336 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:12.026674 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329c287-84c0-49eb-a78f-85f6f2147336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:12.028711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b329c287-84c0-49eb-a78f-85f6f2147336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:12.039681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:12.140317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329c287-84c0-49eb-a78f-85f6f2147336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:12.141908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329c287-84c0-49eb-a78f-85f6f2147336 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:12.161566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 714/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:13.072741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12061af1-42d1-4717-80f4-d861fae2404d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:13.073345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12061af1-42d1-4717-80f4-d861fae2404d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:13.073345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12061af1-42d1-4717-80f4-d861fae2404d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:13.074657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:39:13.081342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511f18-4494-4192-bc16-2d0685e214a9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:13.082544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511f18-4494-4192-bc16-2d0685e214a9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:13.082945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511f18-4494-4192-bc16-2d0685e214a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:13.165286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e0511f18-4494-4192-bc16-2d0685e214a9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:13.166508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e0511f18-4494-4192-bc16-2d0685e214a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:13.243644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:13.279202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:13.435525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12061af1-42d1-4717-80f4-d861fae2404d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:13.443423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-12061af1-42d1-4717-80f4-d861fae2404d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:13.458371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:13.607129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12061af1-42d1-4717-80f4-d861fae2404d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:13.709138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:13.740109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08ab01-d4fd-4dee-a4b3-8937f8544742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:13.740798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08ab01-d4fd-4dee-a4b3-8937f8544742 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:13.741102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08ab01-d4fd-4dee-a4b3-8937f8544742 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:13.741386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08ab01-d4fd-4dee-a4b3-8937f8544742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:13.743172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc08ab01-d4fd-4dee-a4b3-8937f8544742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:13.852300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08ab01-d4fd-4dee-a4b3-8937f8544742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:13.854245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08ab01-d4fd-4dee-a4b3-8937f8544742 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:13.874569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 719/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:14.797167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6884a-1951-4473-8e00-b0fd773104ad None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:14.797853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6884a-1951-4473-8e00-b0fd773104ad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:14.797853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6884a-1951-4473-8e00-b0fd773104ad None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:14.799825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:14.806111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a37722-e37f-4897-8672-9bd14fc2e005 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:14.806916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a37722-e37f-4897-8672-9bd14fc2e005 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:14.806916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a37722-e37f-4897-8672-9bd14fc2e005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:14.890343 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-32a37722-e37f-4897-8672-9bd14fc2e005 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:14.891583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-32a37722-e37f-4897-8672-9bd14fc2e005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:14.948048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:15.011637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:15.170193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6884a-1951-4473-8e00-b0fd773104ad None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:15.171786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ce6884a-1951-4473-8e00-b0fd773104ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:15.327544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6884a-1951-4473-8e00-b0fd773104ad None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:15.387732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:15.449834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb3054-7ea9-41a9-827f-45b367930fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:15.450489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3054-7ea9-41a9-827f-45b367930fa0 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:15.450771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3054-7ea9-41a9-827f-45b367930fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:15.451050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3054-7ea9-41a9-827f-45b367930fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:15.452749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7dbb3054-7ea9-41a9-827f-45b367930fa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:15.552341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb3054-7ea9-41a9-827f-45b367930fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:15.554048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb3054-7ea9-41a9-827f-45b367930fa0 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:15.574292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 717/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:16.503950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c85d7-1cca-4d47-9ffc-af56ef2b03de None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:16.504545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c85d7-1cca-4d47-9ffc-af56ef2b03de None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:16.504958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c85d7-1cca-4d47-9ffc-af56ef2b03de None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:16.507540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:39:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:16.513216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6272-2f63-4b96-9766-b01bc31f6e98 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:16.513726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6272-2f63-4b96-9766-b01bc31f6e98 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:16.513885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6272-2f63-4b96-9766-b01bc31f6e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:16.599391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-707b6272-2f63-4b96-9766-b01bc31f6e98 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:16.600630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-707b6272-2f63-4b96-9766-b01bc31f6e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:16.602836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:16.713056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:16.871168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c85d7-1cca-4d47-9ffc-af56ef2b03de None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:16.872886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1e0c85d7-1cca-4d47-9ffc-af56ef2b03de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:17.035964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c85d7-1cca-4d47-9ffc-af56ef2b03de None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:17.065958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:17.168025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b3237-659b-4f1a-acbb-9761ae08afd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:17.168694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b3237-659b-4f1a-acbb-9761ae08afd3 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:17.169008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b3237-659b-4f1a-acbb-9761ae08afd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:17.169372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b3237-659b-4f1a-acbb-9761ae08afd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:17.171026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ff6b3237-659b-4f1a-acbb-9761ae08afd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:17.268907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b3237-659b-4f1a-acbb-9761ae08afd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:17.270564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b3237-659b-4f1a-acbb-9761ae08afd3 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:17.290724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 722/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:19.347070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7c20f-d656-41f9-a5ec-c15b41a19243 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:19.347792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7c20f-d656-41f9-a5ec-c15b41a19243 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:19.347792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7c20f-d656-41f9-a5ec-c15b41a19243 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:19.349928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:19.356323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef0fc7-f8ac-4d59-b4a1-26a23c26e5e6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:19.356994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef0fc7-f8ac-4d59-b4a1-26a23c26e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:19.356994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef0fc7-f8ac-4d59-b4a1-26a23c26e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:19.401876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:19.430446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-94ef0fc7-f8ac-4d59-b4a1-26a23c26e5e6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:19.431740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-94ef0fc7-f8ac-4d59-b4a1-26a23c26e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:19.545214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:19.585959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7c20f-d656-41f9-a5ec-c15b41a19243 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:19.587566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5de7c20f-d656-41f9-a5ec-c15b41a19243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:19.699857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7c20f-d656-41f9-a5ec-c15b41a19243 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:19.734659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:19.780447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef14ee-3e73-48e6-b357-67968ca385a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:19.781192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef14ee-3e73-48e6-b357-67968ca385a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:19.781420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef14ee-3e73-48e6-b357-67968ca385a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:19.781694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef14ee-3e73-48e6-b357-67968ca385a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:19.783368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-86ef14ee-3e73-48e6-b357-67968ca385a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:19.903479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef14ee-3e73-48e6-b357-67968ca385a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:19.905058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef14ee-3e73-48e6-b357-67968ca385a4 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:19.926799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 720/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:22.269868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7941ce9-bc9d-43c5-a1af-4a3882f4c37d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:22.270451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7941ce9-bc9d-43c5-a1af-4a3882f4c37d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:22.270451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7941ce9-bc9d-43c5-a1af-4a3882f4c37d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:22.271865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:22.278420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10268721-aa75-4113-b9ec-f7b8fc36ac5f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:22.278922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10268721-aa75-4113-b9ec-f7b8fc36ac5f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:22.279084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10268721-aa75-4113-b9ec-f7b8fc36ac5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:22.355377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10268721-aa75-4113-b9ec-f7b8fc36ac5f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:22.356685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10268721-aa75-4113-b9ec-f7b8fc36ac5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:22.372416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:22.469053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:22.509111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7941ce9-bc9d-43c5-a1af-4a3882f4c37d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:22.510636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b7941ce9-bc9d-43c5-a1af-4a3882f4c37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:22.632847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7941ce9-bc9d-43c5-a1af-4a3882f4c37d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:22.646273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:22.714107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0a52c-be98-4c4b-bec9-1f62392457ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:22.714779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0a52c-be98-4c4b-bec9-1f62392457ad service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:22.715065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0a52c-be98-4c4b-bec9-1f62392457ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:22.715347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0a52c-be98-4c4b-bec9-1f62392457ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:22.717090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-45b0a52c-be98-4c4b-bec9-1f62392457ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:22.851191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0a52c-be98-4c4b-bec9-1f62392457ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:22.852820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0a52c-be98-4c4b-bec9-1f62392457ad service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:22.873387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 725/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:39:26.556707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962962ea-7add-453f-bd40-4b60ead91384 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:26.557309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962962ea-7add-453f-bd40-4b60ead91384 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:26.557309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962962ea-7add-453f-bd40-4b60ead91384 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:26.558802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:26.564285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fcf80-64f1-4c17-82d6-64fcee65a9e8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:26.565127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fcf80-64f1-4c17-82d6-64fcee65a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:26.565127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fcf80-64f1-4c17-82d6-64fcee65a9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:26.614751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:26.633996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b84fcf80-64f1-4c17-82d6-64fcee65a9e8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:26.635260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b84fcf80-64f1-4c17-82d6-64fcee65a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:26.736008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:26.775311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962962ea-7add-453f-bd40-4b60ead91384 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:26.776897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-962962ea-7add-453f-bd40-4b60ead91384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:26.894026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962962ea-7add-453f-bd40-4b60ead91384 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:26.938139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:26.977362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1d66a-f730-4669-b4ed-ce671ecea67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:26.978054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d66a-f730-4669-b4ed-ce671ecea67b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:26.978589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d66a-f730-4669-b4ed-ce671ecea67b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:26.978865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d66a-f730-4669-b4ed-ce671ecea67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:26.980626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4c1d66a-f730-4669-b4ed-ce671ecea67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:27.099173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1d66a-f730-4669-b4ed-ce671ecea67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:27.101160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1d66a-f730-4669-b4ed-ce671ecea67b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:27.120933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 723/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:29.376747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c651aa-8073-4223-bbd2-d9c7bfaae825 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:29.377554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c651aa-8073-4223-bbd2-d9c7bfaae825 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:29.377728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c651aa-8073-4223-bbd2-d9c7bfaae825 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:29.379202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:29.384686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad151cdf-6174-4155-9ae1-5b07b49b180e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:29.385257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad151cdf-6174-4155-9ae1-5b07b49b180e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:29.385257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad151cdf-6174-4155-9ae1-5b07b49b180e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:29.453874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ad151cdf-6174-4155-9ae1-5b07b49b180e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:29.455722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad151cdf-6174-4155-9ae1-5b07b49b180e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:29.477822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:29.559572 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:29.599844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c651aa-8073-4223-bbd2-d9c7bfaae825 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:29.601433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24c651aa-8073-4223-bbd2-d9c7bfaae825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:29.711431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c651aa-8073-4223-bbd2-d9c7bfaae825 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:29.731950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:29.789030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5b124-f86f-4034-ab34-84b0a0af4138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:29.789691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b124-f86f-4034-ab34-84b0a0af4138 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:29.789979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b124-f86f-4034-ab34-84b0a0af4138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:29.790260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b124-f86f-4034-ab34-84b0a0af4138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:29.791910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2cc5b124-f86f-4034-ab34-84b0a0af4138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:29.907214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5b124-f86f-4034-ab34-84b0a0af4138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:29.909083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5b124-f86f-4034-ab34-84b0a0af4138 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:29.928303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 728/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:32.146159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895d5ca-38cf-4adc-9616-9445f75af75a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:32.146695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895d5ca-38cf-4adc-9616-9445f75af75a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:32.146695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895d5ca-38cf-4adc-9616-9445f75af75a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:32.148226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:32.153937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb365d-2201-491c-8877-45314f2861b5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:32.154408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb365d-2201-491c-8877-45314f2861b5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:32.154614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb365d-2201-491c-8877-45314f2861b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:32.221814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:32.224698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f8cb365d-2201-491c-8877-45314f2861b5 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:32.225978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f8cb365d-2201-491c-8877-45314f2861b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:32.329742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:32.369484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895d5ca-38cf-4adc-9616-9445f75af75a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:32.371055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4895d5ca-38cf-4adc-9616-9445f75af75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:32.488074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895d5ca-38cf-4adc-9616-9445f75af75a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:32.548248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:32.575952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7dbfd-e5aa-4681-9437-f59016fc6b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:32.576602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7dbfd-e5aa-4681-9437-f59016fc6b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:32.576924 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7dbfd-e5aa-4681-9437-f59016fc6b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:32.577194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7dbfd-e5aa-4681-9437-f59016fc6b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:32.578881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-48c7dbfd-e5aa-4681-9437-f59016fc6b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:32.697120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7dbfd-e5aa-4681-9437-f59016fc6b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:32.698590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7dbfd-e5aa-4681-9437-f59016fc6b7c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:32.717796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 726/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:34.942645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73101c69-b6ff-44bf-97c8-1c7925db3ad8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:34.943280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73101c69-b6ff-44bf-97c8-1c7925db3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:34.943280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73101c69-b6ff-44bf-97c8-1c7925db3ad8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:34.944568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:34.950626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3d17b-93f3-4e04-9444-1d8bd698d5d8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:34.951188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3d17b-93f3-4e04-9444-1d8bd698d5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:34.951188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3d17b-93f3-4e04-9444-1d8bd698d5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:35.023114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d9a3d17b-93f3-4e04-9444-1d8bd698d5d8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:35.024400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d9a3d17b-93f3-4e04-9444-1d8bd698d5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:35.056878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:35.133043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:35.172082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73101c69-b6ff-44bf-97c8-1c7925db3ad8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:35.174275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-73101c69-b6ff-44bf-97c8-1c7925db3ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:35.284053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73101c69-b6ff-44bf-97c8-1c7925db3ad8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:35.315105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:35.365836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216035c-bee3-4aab-a16f-b3959c3476fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:35.366484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216035c-bee3-4aab-a16f-b3959c3476fc service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:35.368171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216035c-bee3-4aab-a16f-b3959c3476fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:35.368171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216035c-bee3-4aab-a16f-b3959c3476fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:35.368697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0216035c-bee3-4aab-a16f-b3959c3476fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:35.490319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216035c-bee3-4aab-a16f-b3959c3476fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:35.491926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216035c-bee3-4aab-a16f-b3959c3476fc service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:35.512370 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 731/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:37.661896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fff49-52cb-43f1-b719-ea4856c01035 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:37.662460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fff49-52cb-43f1-b719-ea4856c01035 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:37.662460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fff49-52cb-43f1-b719-ea4856c01035 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:37.663835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:37.670719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd650-a0bf-43d8-98ce-f65178a6fb64 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:37.671147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd650-a0bf-43d8-98ce-f65178a6fb64 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:37.671274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd650-a0bf-43d8-98ce-f65178a6fb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:37.745092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-058dd650-a0bf-43d8-98ce-f65178a6fb64 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:37.746370 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-058dd650-a0bf-43d8-98ce-f65178a6fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:37.757315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:37.866373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:37.906259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fff49-52cb-43f1-b719-ea4856c01035 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:37.908616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-398fff49-52cb-43f1-b719-ea4856c01035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:38.038568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fff49-52cb-43f1-b719-ea4856c01035 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:38.137126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:38.166473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8064d7-956f-435d-b297-ccd78fe3643a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:38.167406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8064d7-956f-435d-b297-ccd78fe3643a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:38.167815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8064d7-956f-435d-b297-ccd78fe3643a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:38.168210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8064d7-956f-435d-b297-ccd78fe3643a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:38.170839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d8064d7-956f-435d-b297-ccd78fe3643a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:38.341465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8064d7-956f-435d-b297-ccd78fe3643a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:38.343020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8064d7-956f-435d-b297-ccd78fe3643a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:38.365754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 729/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:40.544278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f28b3-a7b6-4037-8f12-3cf6cd96116f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:40.545099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f28b3-a7b6-4037-8f12-3cf6cd96116f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:40.545231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f28b3-a7b6-4037-8f12-3cf6cd96116f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:40.547286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:40.554360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809e9c-733c-4025-ae7e-360ae7fcc65b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:40.554899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809e9c-733c-4025-ae7e-360ae7fcc65b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:40.555034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809e9c-733c-4025-ae7e-360ae7fcc65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:40.651746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-12809e9c-733c-4025-ae7e-360ae7fcc65b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:40.653291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-12809e9c-733c-4025-ae7e-360ae7fcc65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:40.711384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:40.796616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:40.838385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f28b3-a7b6-4037-8f12-3cf6cd96116f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:40.840081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e49f28b3-a7b6-4037-8f12-3cf6cd96116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:40.973630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f28b3-a7b6-4037-8f12-3cf6cd96116f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:41.018161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:41.058611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e185a4-5819-4e9c-a6e0-63d9877878ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:41.059282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e185a4-5819-4e9c-a6e0-63d9877878ee service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:41.059610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e185a4-5819-4e9c-a6e0-63d9877878ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:41.059984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e185a4-5819-4e9c-a6e0-63d9877878ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:41.062694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f5e185a4-5819-4e9c-a6e0-63d9877878ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:41.186324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e185a4-5819-4e9c-a6e0-63d9877878ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:41.187865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e185a4-5819-4e9c-a6e0-63d9877878ee service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:41.210976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 734/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:39:43.841003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab16ef-c9da-488b-9159-0fcdc1b51e47 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:43.841909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab16ef-c9da-488b-9159-0fcdc1b51e47 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:43.841909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab16ef-c9da-488b-9159-0fcdc1b51e47 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:43.843340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:39:43.850654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf5f55-d230-402c-9f37-801d8d1b91cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:43.851291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf5f55-d230-402c-9f37-801d8d1b91cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:43.851493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf5f55-d230-402c-9f37-801d8d1b91cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:43.926749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e6bf5f55-d230-402c-9f37-801d8d1b91cf None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:43.928090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e6bf5f55-d230-402c-9f37-801d8d1b91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:43.949988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:44.073526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:39:44.138040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab16ef-c9da-488b-9159-0fcdc1b51e47 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:44.141091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-44ab16ef-c9da-488b-9159-0fcdc1b51e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:44.313624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab16ef-c9da-488b-9159-0fcdc1b51e47 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:44.397943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:44.405817 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea85559-6aa0-4c47-9f08-ec4b6f8eea6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:44.406401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea85559-6aa0-4c47-9f08-ec4b6f8eea6d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:44.406694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea85559-6aa0-4c47-9f08-ec4b6f8eea6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:44.406975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea85559-6aa0-4c47-9f08-ec4b6f8eea6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:44.409183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ea85559-6aa0-4c47-9f08-ec4b6f8eea6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:44.533388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea85559-6aa0-4c47-9f08-ec4b6f8eea6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:44.534865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea85559-6aa0-4c47-9f08-ec4b6f8eea6d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:44.563133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 732/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:46.789445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc1811-bcfa-44ac-b537-c20f67ef6aae None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.790141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc1811-bcfa-44ac-b537-c20f67ef6aae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.790141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc1811-bcfa-44ac-b537-c20f67ef6aae None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.791444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:39:46.798240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1865a-fb0c-4715-b640-f545464e96b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.799735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1865a-fb0c-4715-b640-f545464e96b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.800126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1865a-fb0c-4715-b640-f545464e96b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.877105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2ba1865a-fb0c-4715-b640-f545464e96b8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:46.878222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ba1865a-fb0c-4715-b640-f545464e96b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:46.933732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:46.994954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:47.035730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc1811-bcfa-44ac-b537-c20f67ef6aae None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:47.037346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fbdc1811-bcfa-44ac-b537-c20f67ef6aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:47.153292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc1811-bcfa-44ac-b537-c20f67ef6aae None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:47.205092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:47.235312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb58a6b-c9be-4de8-be13-3e98b81c9340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:47.236535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb58a6b-c9be-4de8-be13-3e98b81c9340 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:47.236854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb58a6b-c9be-4de8-be13-3e98b81c9340 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:47.237139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb58a6b-c9be-4de8-be13-3e98b81c9340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:47.238968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cfb58a6b-c9be-4de8-be13-3e98b81c9340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:47.364069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb58a6b-c9be-4de8-be13-3e98b81c9340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:47.365484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb58a6b-c9be-4de8-be13-3e98b81c9340 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:47.386391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 737/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:56.992635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=22 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:57.383545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=66 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:07.913772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8591e5-9d36-4ac0-8fca-df1e1c56b38a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:07.914292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8591e5-9d36-4ac0-8fca-df1e1c56b38a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:07.914292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8591e5-9d36-4ac0-8fca-df1e1c56b38a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:07.915706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 734/1471] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:07.921327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e3c0-8ac7-4353-8bbd-692e85adcca0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:07.921744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e3c0-8ac7-4353-8bbd-692e85adcca0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:07.921891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e3c0-8ac7-4353-8bbd-692e85adcca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:07.992428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4bf3e3c0-8ac7-4353-8bbd-692e85adcca0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:07.993679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4bf3e3c0-8ac7-4353-8bbd-692e85adcca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.099737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 738/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:08.151355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8f9e-4443-4b2e-88f8-94682900a385 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:08.151647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8f9e-4443-4b2e-88f8-94682900a385 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:08.151931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8f9e-4443-4b2e-88f8-94682900a385 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:08.153379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 735/1473] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 18 10:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:08.159065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994931a-e7d2-4940-a865-03bf9d2856b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:08.159406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994931a-e7d2-4940-a865-03bf9d2856b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:08.159646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994931a-e7d2-4940-a865-03bf9d2856b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:08.238954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1994931a-e7d2-4940-a865-03bf9d2856b9 None None] MFA Rules not processed for user `80535e7da62e4790931fd74000653411`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:08.240161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1994931a-e7d2-4940-a865-03bf9d2856b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.262801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:08.323468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 739/1474] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 18 10:40:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:08.331172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e8f9e-4443-4b2e-88f8-94682900a385 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:08.332725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0e2e8f9e-4443-4b2e-88f8-94682900a385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.444350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e8f9e-4443-4b2e-88f8-94682900a385 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:08.521788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da452d9-4c41-4080-ae48-82b36e9cc59c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:08.522372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da452d9-4c41-4080-ae48-82b36e9cc59c service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:08.522664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da452d9-4c41-4080-ae48-82b36e9cc59c service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:08.522943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da452d9-4c41-4080-ae48-82b36e9cc59c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:08.524600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7da452d9-4c41-4080-ae48-82b36e9cc59c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.582432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:08.638560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da452d9-4c41-4080-ae48-82b36e9cc59c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:08.640036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da452d9-4c41-4080-ae48-82b36e9cc59c service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:08.661916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 736/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:10.890827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf6598-7666-4202-912f-dee3bb79495a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:10.891392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf6598-7666-4202-912f-dee3bb79495a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:10.891392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf6598-7666-4202-912f-dee3bb79495a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:10.892948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 740/1476] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:10.898717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff52c07-416f-4652-a2ef-7cffc7003162 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:10.899191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff52c07-416f-4652-a2ef-7cffc7003162 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:10.899286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff52c07-416f-4652-a2ef-7cffc7003162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:10.967223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fff52c07-416f-4652-a2ef-7cffc7003162 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:10.968459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fff52c07-416f-4652-a2ef-7cffc7003162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:11.068972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 737/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:11.092074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf6598-7666-4202-912f-dee3bb79495a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:11.093730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-01cf6598-7666-4202-912f-dee3bb79495a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:11.224719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf6598-7666-4202-912f-dee3bb79495a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:11.276697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:11.323899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72808ac0-9304-4c0f-8b18-f74a30acc0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:11.324480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72808ac0-9304-4c0f-8b18-f74a30acc0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:11.324763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72808ac0-9304-4c0f-8b18-f74a30acc0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:11.325104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72808ac0-9304-4c0f-8b18-f74a30acc0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:11.325849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72808ac0-9304-4c0f-8b18-f74a30acc0c8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:11.327654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-72808ac0-9304-4c0f-8b18-f74a30acc0c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:40:11.327654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:40:11.327654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:40:11.329043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72808ac0-9304-4c0f-8b18-f74a30acc0c8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:11.348156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 741/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 18 10:40:11 2022] GET /identity/v3/projects => generated 2553 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:13.446826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb37c2-0180-470f-aa01-0249fa7a8f6e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:13.447517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb37c2-0180-470f-aa01-0249fa7a8f6e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:13.447517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb37c2-0180-470f-aa01-0249fa7a8f6e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:13.448946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 738/1479] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:13.454688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd43a3c-d147-42c2-b9ef-543a320bad43 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:13.455190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd43a3c-d147-42c2-b9ef-543a320bad43 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:13.455190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd43a3c-d147-42c2-b9ef-543a320bad43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:13.523824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5cd43a3c-d147-42c2-b9ef-543a320bad43 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:13.525078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5cd43a3c-d147-42c2-b9ef-543a320bad43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:13.750111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 742/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:13.796165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb37c2-0180-470f-aa01-0249fa7a8f6e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:13.798410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fbb37c2-0180-470f-aa01-0249fa7a8f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:13.809316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:13.951226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb37c2-0180-470f-aa01-0249fa7a8f6e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:14.033169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:14.066570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aac0c-a3fa-4abe-b623-6ec727d57f1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:14.067454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aac0c-a3fa-4abe-b623-6ec727d57f1b service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:14.068034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aac0c-a3fa-4abe-b623-6ec727d57f1b service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:14.068580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aac0c-a3fa-4abe-b623-6ec727d57f1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:14.070298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-466aac0c-a3fa-4abe-b623-6ec727d57f1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:14.163612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aac0c-a3fa-4abe-b623-6ec727d57f1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:14.165295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aac0c-a3fa-4abe-b623-6ec727d57f1b service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:14.184966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 739/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:16.485605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f58c72-3475-4a6a-bd46-91283a8c7309 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:16.486184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f58c72-3475-4a6a-bd46-91283a8c7309 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:16.486184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f58c72-3475-4a6a-bd46-91283a8c7309 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:16.487558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 743/1482] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:16.493200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a496ce-b9ff-4ce5-ad9e-02b58136138f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:16.493657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a496ce-b9ff-4ce5-ad9e-02b58136138f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:16.493770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a496ce-b9ff-4ce5-ad9e-02b58136138f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:16.562436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6a496ce-b9ff-4ce5-ad9e-02b58136138f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:16.564577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6a496ce-b9ff-4ce5-ad9e-02b58136138f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:16.614116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:16.667387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 740/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:16.704432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f58c72-3475-4a6a-bd46-91283a8c7309 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:16.708090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c7f58c72-3475-4a6a-bd46-91283a8c7309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:16.730997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:16.815856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f58c72-3475-4a6a-bd46-91283a8c7309 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:16.892426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c33f6-a4da-4cbd-9619-0b28414b246b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:16.893123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c33f6-a4da-4cbd-9619-0b28414b246b service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:16.893413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c33f6-a4da-4cbd-9619-0b28414b246b service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:16.893692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c33f6-a4da-4cbd-9619-0b28414b246b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:16.895437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c0c33f6-a4da-4cbd-9619-0b28414b246b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:16.975418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:17.011371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c33f6-a4da-4cbd-9619-0b28414b246b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:17.012855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c33f6-a4da-4cbd-9619-0b28414b246b service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:17.032397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 744/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:19.157514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c901e-a6f0-43b7-8cdb-2f6b4b8a6997 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:19.158094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c901e-a6f0-43b7-8cdb-2f6b4b8a6997 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:19.158094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c901e-a6f0-43b7-8cdb-2f6b4b8a6997 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:19.159412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 741/1485] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:19.165118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f992fc-0ea7-44e8-bbe3-2375b7ce6915 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:19.165576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f992fc-0ea7-44e8-bbe3-2375b7ce6915 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:19.165697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f992fc-0ea7-44e8-bbe3-2375b7ce6915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:19.232502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c6f992fc-0ea7-44e8-bbe3-2375b7ce6915 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:19.233744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6f992fc-0ea7-44e8-bbe3-2375b7ce6915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:19.334131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 745/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:19.372706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c901e-a6f0-43b7-8cdb-2f6b4b8a6997 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:19.374350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f3c901e-a6f0-43b7-8cdb-2f6b4b8a6997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:19.482618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c901e-a6f0-43b7-8cdb-2f6b4b8a6997 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:19.558886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:19.560816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458f64-3a9b-49c1-8b26-e98024b14755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:19.561487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458f64-3a9b-49c1-8b26-e98024b14755 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:19.561834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458f64-3a9b-49c1-8b26-e98024b14755 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:19.562118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458f64-3a9b-49c1-8b26-e98024b14755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:19.564344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f458f64-3a9b-49c1-8b26-e98024b14755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:19.682119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f458f64-3a9b-49c1-8b26-e98024b14755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:19.683582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f458f64-3a9b-49c1-8b26-e98024b14755 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:19.703185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 742/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:21.943988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8238-d4be-47a9-a795-1530069c1440 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:21.944563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8238-d4be-47a9-a795-1530069c1440 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:21.944563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8238-d4be-47a9-a795-1530069c1440 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:21.946010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 746/1488] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:21.951563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ceb6c-93d7-4349-8937-7c4481256e0b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:21.952055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ceb6c-93d7-4349-8937-7c4481256e0b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:21.952151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ceb6c-93d7-4349-8937-7c4481256e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:22.021200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bb9ceb6c-93d7-4349-8937-7c4481256e0b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:22.022425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb9ceb6c-93d7-4349-8937-7c4481256e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:22.077859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:22.123590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 743/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:22.166056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a8238-d4be-47a9-a795-1530069c1440 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:22.167633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-899a8238-d4be-47a9-a795-1530069c1440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:22.216641 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:22.290749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a8238-d4be-47a9-a795-1530069c1440 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:22.368595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b3747-f9b9-43fb-9f2c-f7fda40d2b5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:22.369351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b3747-f9b9-43fb-9f2c-f7fda40d2b5c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:22.369634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b3747-f9b9-43fb-9f2c-f7fda40d2b5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:22.369896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b3747-f9b9-43fb-9f2c-f7fda40d2b5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:22.371545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d18b3747-f9b9-43fb-9f2c-f7fda40d2b5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:22.468249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:22.499425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b3747-f9b9-43fb-9f2c-f7fda40d2b5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:22.501546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b3747-f9b9-43fb-9f2c-f7fda40d2b5c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:22.520096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 747/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:25.238228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc8acf-df93-423b-8915-5b106ffb76a5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:25.238826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc8acf-df93-423b-8915-5b106ffb76a5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:25.239201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc8acf-df93-423b-8915-5b106ffb76a5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:25.241684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 744/1491] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 18 10:40:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:25.247683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a9338-2153-4ac4-9759-8f68f4145570 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:25.248227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a9338-2153-4ac4-9759-8f68f4145570 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:25.248332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a9338-2153-4ac4-9759-8f68f4145570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:25.326207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7c2a9338-2153-4ac4-9759-8f68f4145570 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:25.327575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7c2a9338-2153-4ac4-9759-8f68f4145570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:25.439218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 748/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 18 10:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:25.508706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc8acf-df93-423b-8915-5b106ffb76a5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:25.511452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-00cc8acf-df93-423b-8915-5b106ffb76a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:25.707309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc8acf-df93-423b-8915-5b106ffb76a5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:25.806969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f7dd4-ffaf-4a8e-bb6f-3d12d001adb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:25.808005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f7dd4-ffaf-4a8e-bb6f-3d12d001adb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:25.809130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f7dd4-ffaf-4a8e-bb6f-3d12d001adb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:25.809130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f7dd4-ffaf-4a8e-bb6f-3d12d001adb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:25.810871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-382f7dd4-ffaf-4a8e-bb6f-3d12d001adb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:25.824584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:26.015643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f7dd4-ffaf-4a8e-bb6f-3d12d001adb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:26.018938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f7dd4-ffaf-4a8e-bb6f-3d12d001adb1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:26.052650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 745/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:35.436896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=82 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:36.049000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=50 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:41.947894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb975b-b5cd-42b4-adb6-50e8cf7671ed None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:41.948645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb975b-b5cd-42b4-adb6-50e8cf7671ed None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:41.948645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb975b-b5cd-42b4-adb6-50e8cf7671ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:42.064616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-92bb975b-b5cd-42b4-adb6-50e8cf7671ed None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:42.067095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92bb975b-b5cd-42b4-adb6-50e8cf7671ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:42.259295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 749/1494] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:42.269161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf17ef-4743-4ab1-870f-3abe77fd2e29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:42.270011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf17ef-4743-4ab1-870f-3abe77fd2e29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:42.270011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf17ef-4743-4ab1-870f-3abe77fd2e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:42.292875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1fbf17ef-4743-4ab1-870f-3abe77fd2e29 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:42.294441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fbf17ef-4743-4ab1-870f-3abe77fd2e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:42.305204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 746/1495] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:42.310965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb975b-b5cd-42b4-adb6-50e8cf7671ed None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:42.327958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb975b-b5cd-42b4-adb6-50e8cf7671ed None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:42.334454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:42.335583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:42.335976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:42.336344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:42.337259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:42.337617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:42.338863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:42.338863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:42:42.338863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:42:42.340804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:42.362934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 750/1496] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:42.397090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf17ef-4743-4ab1-870f-3abe77fd2e29 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:42.414158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf17ef-4743-4ab1-870f-3abe77fd2e29 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:42.422954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:42.428895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:42.428895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:42.428895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:42.428895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:42.430603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:42.430603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jul 18 10:42:42.430603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: Jul 18 10:42:42.433183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:42.663825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 747/1497] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:42 2022] POST /identity/v3/projects => generated 379 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:42.671604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:42.690328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22487b1a-f8b9-43e5-a114-67369ddb2da1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:42.791704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:42.802291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:42.803513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:42.804035 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:42.804493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:42.805480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:42.806795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:42.806795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:42:42.806795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:42:42.809278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:42.936375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 751/1498] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:42 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:42.943583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:42.958861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94b131-edf5-403d-9b95-332ea0ab442b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:42.967865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:42.969124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:42.969654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:42.970158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:42.971630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:42.973314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:42.973314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jul 18 10:42:42.973314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: Jul 18 10:42:42.975046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:43.000966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 748/1499] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:42:43.007584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:43.025568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a79e5-995b-4026-8800-16d538a50b34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:43.031818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e3c81-c05e-4912-b6d8-df33c4fb51ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:43.032527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e3c81-c05e-4912-b6d8-df33c4fb51ba admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.032848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e3c81-c05e-4912-b6d8-df33c4fb51ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.033132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e3c81-c05e-4912-b6d8-df33c4fb51ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.033911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e3c81-c05e-4912-b6d8-df33c4fb51ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:43.034976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-e79e3c81-c05e-4912-b6d8-df33c4fb51ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:43.034976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:42:43.034976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:42:43.036034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e3c81-c05e-4912-b6d8-df33c4fb51ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:43.056700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 752/1500] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:43.063690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:43.080195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052963-e188-4cae-b41e-491a98e8a68b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:43.086154 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7306a5-b1ca-423e-9181-2db7b8f6df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:43.087400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7306a5-b1ca-423e-9181-2db7b8f6df0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.087863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7306a5-b1ca-423e-9181-2db7b8f6df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.088240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7306a5-b1ca-423e-9181-2db7b8f6df0f admin admin] PATH_INFO: `/v3/projects/b52bfaf0943e418db40b6eff5bcb5df8/users/cd0e357013cb4fa3b1a18ee504323c60/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.110677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7306a5-b1ca-423e-9181-2db7b8f6df0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52bfaf0943e418db40b6eff5bcb5df8, user_id=cd0e357013cb4fa3b1a18ee504323c60, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:43.111783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-ae7306a5-b1ca-423e-9181-2db7b8f6df0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:43.111783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jul 18 10:42:43.111783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: Jul 18 10:42:43.113883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7306a5-b1ca-423e-9181-2db7b8f6df0f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:43.317008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 749/1501] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:43 2022] PUT /identity/v3/projects/b52bfaf0943e418db40b6eff5bcb5df8/users/cd0e357013cb4fa3b1a18ee504323c60/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:42:43.343562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572317e-6010-4cfe-82ef-f5960ac0c443 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.343894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572317e-6010-4cfe-82ef-f5960ac0c443 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.344238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572317e-6010-4cfe-82ef-f5960ac0c443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.376718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4572317e-6010-4cfe-82ef-f5960ac0c443 None None] MFA Rules not processed for user `cd0e357013cb4fa3b1a18ee504323c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:43.378036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4572317e-6010-4cfe-82ef-f5960ac0c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:43.468023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 753/1502] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:42:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:43.483527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30fce-dd3c-4e16-bda7-dcdee86b8280 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.483862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30fce-dd3c-4e16-bda7-dcdee86b8280 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.484172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30fce-dd3c-4e16-bda7-dcdee86b8280 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.485762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 750/1503] 10.222.0.49 () {54 vars in 964 bytes} [Mon Jul 18 10:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:42:43.493363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5245a-2863-4f3f-accb-e302339be753 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.493705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5245a-2863-4f3f-accb-e302339be753 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.494034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5245a-2863-4f3f-accb-e302339be753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.594360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f6c5245a-2863-4f3f-accb-e302339be753 None None] MFA Rules not processed for user `62e462a1b125428f9893459d5d965ced`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:43.595643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f6c5245a-2863-4f3f-accb-e302339be753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:43.741473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 754/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Jul 18 10:42:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:43.755958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b30fce-dd3c-4e16-bda7-dcdee86b8280 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:43.775109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b30fce-dd3c-4e16-bda7-dcdee86b8280 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:43.782293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:43.783554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.784074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.784582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.801728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:43.804623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:43.822483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 751/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:43.884273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac415e80-4dce-478c-abf1-f9ebac9ea380 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.884728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac415e80-4dce-478c-abf1-f9ebac9ea380 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.885182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac415e80-4dce-478c-abf1-f9ebac9ea380 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.888062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 755/1506] 10.222.0.49 () {52 vars in 829 bytes} [Mon Jul 18 10:42:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:42:43.893408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:43.983132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce633-e6bc-4389-bc03-e5445edf1865 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:43.988744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:43.989764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:43.990118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:43.990512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] PATH_INFO: `/v3/users/cd0e357013cb4fa3b1a18ee504323c60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:43.992369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd0e357013cb4fa3b1a18ee504323c60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:43.993549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:43.993549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jul 18 10:42:43.993549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: Jul 18 10:42:43.995028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:44.252985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd0e357013cb4fa3b1a18ee504323c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:44.285164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd0e357013cb4fa3b1a18ee504323c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:44.358903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd0e357013cb4fa3b1a18ee504323c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:44.359965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd0e357013cb4fa3b1a18ee504323c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:44.361251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd0e357013cb4fa3b1a18ee504323c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:44.377615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:44.400518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 752/1507] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:42:43 2022] DELETE /identity/v3/users/cd0e357013cb4fa3b1a18ee504323c60 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:44.407124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:44.407977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:44.407977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:44.433870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:44.435882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:44.454499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:44.533733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 756/1508] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:42:44.554198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:44.559765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:44.704693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f9f3-97e1-4402-8d92-180f1cb190f3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:44.711402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086663a-fb8d-4d0e-8fff-bbd0abfc7bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:44.712597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086663a-fb8d-4d0e-8fff-bbd0abfc7bfe service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:44.713131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086663a-fb8d-4d0e-8fff-bbd0abfc7bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:44.713615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086663a-fb8d-4d0e-8fff-bbd0abfc7bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:44.731018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086663a-fb8d-4d0e-8fff-bbd0abfc7bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:44.733598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086663a-fb8d-4d0e-8fff-bbd0abfc7bfe service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:44.758177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 753/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:42:45.635032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:45.638673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:45.668530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebfd01-b743-466b-8523-74fed1df0a20 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:45.670688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:45.672320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:45.672898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:45.673432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] PATH_INFO: `/v3/projects/b52bfaf0943e418db40b6eff5bcb5df8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:45.676011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52bfaf0943e418db40b6eff5bcb5df8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:45.678047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:45.678047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:42:45.678047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:42:45.680493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:45.973946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52bfaf0943e418db40b6eff5bcb5df8', 'request_id': 'req-79551315-d9b0-42e8-8a70-af5de34eae7d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:46.006134 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52bfaf0943e418db40b6eff5bcb5df8', 'request_id': 'req-79551315-d9b0-42e8-8a70-af5de34eae7d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:46.135240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52bfaf0943e418db40b6eff5bcb5df8', 'request_id': 'req-79551315-d9b0-42e8-8a70-af5de34eae7d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:46.158656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] The token cache is being invalidate because project b52bfaf0943e418db40b6eff5bcb5df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:42:46.159144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:46.160375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-79551315-d9b0-42e8-8a70-af5de34eae7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:46.163817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 757/1510] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:42:45 2022] DELETE /identity/v3/projects/b52bfaf0943e418db40b6eff5bcb5df8 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:54.743797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=38 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:56.150204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:48.811657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2371a60-443c-4987-8f97-b5b8f3b17cd2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:48.812831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2371a60-443c-4987-8f97-b5b8f3b17cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:48.812831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2371a60-443c-4987-8f97-b5b8f3b17cd2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:48.814306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:44:48.821277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40c86e-5122-42ba-a454-75060e6f7842 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:48.822029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40c86e-5122-42ba-a454-75060e6f7842 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:48.822029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40c86e-5122-42ba-a454-75060e6f7842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:48.848210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cd40c86e-5122-42ba-a454-75060e6f7842 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:48.850032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd40c86e-5122-42ba-a454-75060e6f7842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:48.935072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:49.135576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2371a60-443c-4987-8f97-b5b8f3b17cd2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:49.141452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d2371a60-443c-4987-8f97-b5b8f3b17cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:49.303205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2371a60-443c-4987-8f97-b5b8f3b17cd2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:49.310258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d24c-29f1-4f6a-8e59-13efffde9026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:49.310949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d24c-29f1-4f6a-8e59-13efffde9026 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:49.311236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d24c-29f1-4f6a-8e59-13efffde9026 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:49.311518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d24c-29f1-4f6a-8e59-13efffde9026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:49.323145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455d24c-29f1-4f6a-8e59-13efffde9026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:49.324879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455d24c-29f1-4f6a-8e59-13efffde9026 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:49.336284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 755/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:50.023500 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8ce7a-082c-4bf5-b57d-9157896b308f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:50.024599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8ce7a-082c-4bf5-b57d-9157896b308f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:50.024599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8ce7a-082c-4bf5-b57d-9157896b308f None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:50.027143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 759/1514] 10.222.0.52 () {54 vars in 925 bytes} [Mon Jul 18 10:44:50 2022] GET /identity/v3 => generated 255 bytes in 6 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:44:50.053990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a97488-70b8-41b2-9440-3a146c443da7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:50.055543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a97488-70b8-41b2-9440-3a146c443da7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:50.056281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a97488-70b8-41b2-9440-3a146c443da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:50.166909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b8a97488-70b8-41b2-9440-3a146c443da7 None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:50.169776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8a97488-70b8-41b2-9440-3a146c443da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:50.326733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 756/1515] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Jul 18 10:44:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:50.472325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa78268-c3d8-43e9-bc55-6958bd9466c8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:50.472796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa78268-c3d8-43e9-bc55-6958bd9466c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:50.473223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa78268-c3d8-43e9-bc55-6958bd9466c8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:50.475429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:44:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:44:50.483171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930445e-5abb-410a-9ab2-4e7a30b97bd9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:50.483626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930445e-5abb-410a-9ab2-4e7a30b97bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:50.484230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930445e-5abb-410a-9ab2-4e7a30b97bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:50.506518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f930445e-5abb-410a-9ab2-4e7a30b97bd9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:50.508479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f930445e-5abb-410a-9ab2-4e7a30b97bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:50.518364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:50.700475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa78268-c3d8-43e9-bc55-6958bd9466c8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:50.717147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa78268-c3d8-43e9-bc55-6958bd9466c8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:50.723351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328abf6a-42dd-43b3-96e8-f3fa8be6d0ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:50.724277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328abf6a-42dd-43b3-96e8-f3fa8be6d0ea service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:50.724557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328abf6a-42dd-43b3-96e8-f3fa8be6d0ea service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:50.724859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328abf6a-42dd-43b3-96e8-f3fa8be6d0ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:50.735712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328abf6a-42dd-43b3-96e8-f3fa8be6d0ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:50.737408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328abf6a-42dd-43b3-96e8-f3fa8be6d0ea service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:50.748623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 761/1518] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:54.594757 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8bf12-3883-420b-aede-ec1cbf46fd79 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.595901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8bf12-3883-420b-aede-ec1cbf46fd79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.595901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8bf12-3883-420b-aede-ec1cbf46fd79 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.596833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 758/1519] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 18 10:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:44:54.605140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf7fd0-2a74-4eb8-8e7e-40c82ebc7ef3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.605997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf7fd0-2a74-4eb8-8e7e-40c82ebc7ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.606371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf7fd0-2a74-4eb8-8e7e-40c82ebc7ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.645490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-13bf7fd0-2a74-4eb8-8e7e-40c82ebc7ef3 None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:54.648593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-13bf7fd0-2a74-4eb8-8e7e-40c82ebc7ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.665353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 762/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 18 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:54.674767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8bf12-3883-420b-aede-ec1cbf46fd79 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.692258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8bf12-3883-420b-aede-ec1cbf46fd79 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.699770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35296d1c-ae92-4b06-b14e-4952079663aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.701017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35296d1c-ae92-4b06-b14e-4952079663aa service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.701568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35296d1c-ae92-4b06-b14e-4952079663aa service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.701890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35296d1c-ae92-4b06-b14e-4952079663aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.718584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35296d1c-ae92-4b06-b14e-4952079663aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.720494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35296d1c-ae92-4b06-b14e-4952079663aa service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.737489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 759/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:02.058823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65edd7c-58da-49b3-9cc7-1e9621e4a620 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:02.059389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65edd7c-58da-49b3-9cc7-1e9621e4a620 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:02.059727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65edd7c-58da-49b3-9cc7-1e9621e4a620 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:02.061966 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:45:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:45:02.067783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edead3-f37a-44ec-a37d-62920ad813bf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:02.068398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edead3-f37a-44ec-a37d-62920ad813bf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:02.068398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edead3-f37a-44ec-a37d-62920ad813bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:02.075881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:02.090588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-30edead3-f37a-44ec-a37d-62920ad813bf None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:02.091844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-30edead3-f37a-44ec-a37d-62920ad813bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:02.103034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:02.265285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65edd7c-58da-49b3-9cc7-1e9621e4a620 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:02.283382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65edd7c-58da-49b3-9cc7-1e9621e4a620 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:02.289771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29108-2739-41e5-9eab-ba510e360892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:02.290619 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29108-2739-41e5-9eab-ba510e360892 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:02.290898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29108-2739-41e5-9eab-ba510e360892 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:02.291197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29108-2739-41e5-9eab-ba510e360892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:02.300488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff29108-2739-41e5-9eab-ba510e360892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:02.302140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff29108-2739-41e5-9eab-ba510e360892 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:02.313825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 764/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:03.163299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a4b2-5c28-470a-b7bc-e7552027f8d2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:03.164420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a4b2-5c28-470a-b7bc-e7552027f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:03.164420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a4b2-5c28-470a-b7bc-e7552027f8d2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:03.165913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:45:03.171462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a7771-e872-45d3-9efa-c82f28d204f3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:03.171964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a7771-e872-45d3-9efa-c82f28d204f3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:03.172062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a7771-e872-45d3-9efa-c82f28d204f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:03.191079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1f1a7771-e872-45d3-9efa-c82f28d204f3 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:03.192310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1f1a7771-e872-45d3-9efa-c82f28d204f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:03.201381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:03.356996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6a4b2-5c28-470a-b7bc-e7552027f8d2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:03.372059 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6a4b2-5c28-470a-b7bc-e7552027f8d2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:03.377802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b889aef-a9d8-48d7-b075-427a2a9a68f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:03.378467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889aef-a9d8-48d7-b075-427a2a9a68f2 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:03.378759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889aef-a9d8-48d7-b075-427a2a9a68f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:03.379042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889aef-a9d8-48d7-b075-427a2a9a68f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:03.388352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b889aef-a9d8-48d7-b075-427a2a9a68f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:03.390101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b889aef-a9d8-48d7-b075-427a2a9a68f2 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:03.401512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 762/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:07.590901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3460a-504d-4947-865d-eee893c091aa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:07.591489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3460a-504d-4947-865d-eee893c091aa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:07.591489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3460a-504d-4947-865d-eee893c091aa None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:07.592870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:45:07.599047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac4db4-194e-468f-a9d8-38e9d7d20a0e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:07.599604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac4db4-194e-468f-a9d8-38e9d7d20a0e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:07.599604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac4db4-194e-468f-a9d8-38e9d7d20a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:07.623030 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7fac4db4-194e-468f-a9d8-38e9d7d20a0e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:07.624276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fac4db4-194e-468f-a9d8-38e9d7d20a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:07.632952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:07.672214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3460a-504d-4947-865d-eee893c091aa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:07.674066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2cc3460a-504d-4947-865d-eee893c091aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:07.736488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3460a-504d-4947-865d-eee893c091aa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:07.740143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f1fff-4639-4005-98d2-619940e24937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:07.740818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f1fff-4639-4005-98d2-619940e24937 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:07.741108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f1fff-4639-4005-98d2-619940e24937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:07.741575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f1fff-4639-4005-98d2-619940e24937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:07.752100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f1fff-4639-4005-98d2-619940e24937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:07.753618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f1fff-4639-4005-98d2-619940e24937 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:07.764217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 767/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:09.842913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd794d-96e5-4acf-a28f-15dd16de5079 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:09.843343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd794d-96e5-4acf-a28f-15dd16de5079 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:09.843547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd794d-96e5-4acf-a28f-15dd16de5079 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:09.845244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:45:09.852223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5d9f-909f-45cc-b728-843df9a77d3d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:09.858161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5d9f-909f-45cc-b728-843df9a77d3d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:09.858161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5d9f-909f-45cc-b728-843df9a77d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:09.872904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ed5d5d9f-909f-45cc-b728-843df9a77d3d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:09.874171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed5d5d9f-909f-45cc-b728-843df9a77d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:09.883133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:09.923485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd794d-96e5-4acf-a28f-15dd16de5079 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:09.936971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd794d-96e5-4acf-a28f-15dd16de5079 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:09.940666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca594690-cea9-4dd2-bd78-90170d7136b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:09.941365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca594690-cea9-4dd2-bd78-90170d7136b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:09.941655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca594690-cea9-4dd2-bd78-90170d7136b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:09.941935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca594690-cea9-4dd2-bd78-90170d7136b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:09.952956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca594690-cea9-4dd2-bd78-90170d7136b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:09.954450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca594690-cea9-4dd2-bd78-90170d7136b1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:09.966427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 765/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:45:13.618717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241289e-daef-4835-a054-ade59026d30a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:13.619269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241289e-daef-4835-a054-ade59026d30a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:13.619717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241289e-daef-4835-a054-ade59026d30a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:13.621399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:45:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:45:13.627495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daab82-04bf-4586-90f4-fd3211402fa9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:13.628017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daab82-04bf-4586-90f4-fd3211402fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:13.628141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daab82-04bf-4586-90f4-fd3211402fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:13.772758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b0daab82-04bf-4586-90f4-fd3211402fa9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:13.774240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b0daab82-04bf-4586-90f4-fd3211402fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:13.782481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:14.128303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241289e-daef-4835-a054-ade59026d30a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:14.130029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b241289e-daef-4835-a054-ade59026d30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:14.193268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241289e-daef-4835-a054-ade59026d30a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:14.198177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff23d-dcb1-4c54-baec-488ad981df36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:14.198867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff23d-dcb1-4c54-baec-488ad981df36 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:14.199139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff23d-dcb1-4c54-baec-488ad981df36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:14.199414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff23d-dcb1-4c54-baec-488ad981df36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:14.209599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcff23d-dcb1-4c54-baec-488ad981df36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:14.212104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcff23d-dcb1-4c54-baec-488ad981df36 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:14.217853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:14.222862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 770/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:16.350323 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f16ce-e20b-4918-bdc1-8bd233ba1d1b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.350927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f16ce-e20b-4918-bdc1-8bd233ba1d1b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.351836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f16ce-e20b-4918-bdc1-8bd233ba1d1b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.352580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:45:16.362815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33481c97-575d-4780-9786-359c267d14fd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.363377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33481c97-575d-4780-9786-359c267d14fd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.363377 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33481c97-575d-4780-9786-359c267d14fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.385752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-33481c97-575d-4780-9786-359c267d14fd None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:16.387011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-33481c97-575d-4780-9786-359c267d14fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:16.398766 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:16.731017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f16ce-e20b-4918-bdc1-8bd233ba1d1b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.732722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-192f16ce-e20b-4918-bdc1-8bd233ba1d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:16.746404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f16ce-e20b-4918-bdc1-8bd233ba1d1b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.749987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aee47c-e0af-41a3-8f16-ee1da3995ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.750646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aee47c-e0af-41a3-8f16-ee1da3995ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.750929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aee47c-e0af-41a3-8f16-ee1da3995ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.751797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aee47c-e0af-41a3-8f16-ee1da3995ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.761894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aee47c-e0af-41a3-8f16-ee1da3995ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.763952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aee47c-e0af-41a3-8f16-ee1da3995ef5 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.776061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 768/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:26.377562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:26.770624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=26 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:53.001222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5f5e-c9f8-433b-9b35-18794514f4b8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.002471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5f5e-c9f8-433b-9b35-18794514f4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.002471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5f5e-c9f8-433b-9b35-18794514f4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.048285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f0ac5f5e-c9f8-433b-9b35-18794514f4b8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.049712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0ac5f5e-c9f8-433b-9b35-18794514f4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.063894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 772/1540] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.080720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0715be7-3364-4e6c-9833-f1ee0d1d66fb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.081855 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0715be7-3364-4e6c-9833-f1ee0d1d66fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.083116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0715be7-3364-4e6c-9833-f1ee0d1d66fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.113291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d0715be7-3364-4e6c-9833-f1ee0d1d66fb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.115542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0715be7-3364-4e6c-9833-f1ee0d1d66fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.128847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 769/1541] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.135028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5f5e-c9f8-433b-9b35-18794514f4b8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.153263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5f5e-c9f8-433b-9b35-18794514f4b8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.162391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b99967-9f63-4f93-b5bf-4e4f47144292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.163093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99967-9f63-4f93-b5bf-4e4f47144292 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.163385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99967-9f63-4f93-b5bf-4e4f47144292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.163660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99967-9f63-4f93-b5bf-4e4f47144292 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.164414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99967-9f63-4f93-b5bf-4e4f47144292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:53.164734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99967-9f63-4f93-b5bf-4e4f47144292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.166567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99967-9f63-4f93-b5bf-4e4f47144292 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.187754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 773/1542] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:53.197426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0715be7-3364-4e6c-9833-f1ee0d1d66fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.216275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0715be7-3364-4e6c-9833-f1ee0d1d66fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.226072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0683ad5-4adc-4d79-bf3f-05e609efb90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.227173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0683ad5-4adc-4d79-bf3f-05e609efb90f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.227687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0683ad5-4adc-4d79-bf3f-05e609efb90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.228197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0683ad5-4adc-4d79-bf3f-05e609efb90f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.229154 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0683ad5-4adc-4d79-bf3f-05e609efb90f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.231382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0683ad5-4adc-4d79-bf3f-05e609efb90f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.273029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74160-fd0a-4856-b772-2153fb3c4bce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.273636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74160-fd0a-4856-b772-2153fb3c4bce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.274011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74160-fd0a-4856-b772-2153fb3c4bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.312318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0fc74160-fd0a-4856-b772-2153fb3c4bce None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.315043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0fc74160-fd0a-4856-b772-2153fb3c4bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.331447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 774/1543] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.334429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceb872-11ed-4d18-bf04-2ca3b02eb00a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.334770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceb872-11ed-4d18-bf04-2ca3b02eb00a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.334998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceb872-11ed-4d18-bf04-2ca3b02eb00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.345704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 770/1544] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:53.348532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146d2df-40ca-4b98-a21a-7823ce27c89a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.349434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146d2df-40ca-4b98-a21a-7823ce27c89a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.350084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146d2df-40ca-4b98-a21a-7823ce27c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.374031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fbceb872-11ed-4d18-bf04-2ca3b02eb00a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.375662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fbceb872-11ed-4d18-bf04-2ca3b02eb00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.388950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0146d2df-40ca-4b98-a21a-7823ce27c89a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.393164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0146d2df-40ca-4b98-a21a-7823ce27c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.520652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 771/1545] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.522808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c25c7d-a9a3-4524-87fc-9ba0f58c6853 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.523107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c25c7d-a9a3-4524-87fc-9ba0f58c6853 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.523386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c25c7d-a9a3-4524-87fc-9ba0f58c6853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.545233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a0c25c7d-a9a3-4524-87fc-9ba0f58c6853 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.546563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0c25c7d-a9a3-4524-87fc-9ba0f58c6853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.556988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 772/1546] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.558160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c25c7d-a9a3-4524-87fc-9ba0f58c6853 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.571139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c25c7d-a9a3-4524-87fc-9ba0f58c6853 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.571546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 775/1547] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.573017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fe36-e72d-41d8-b7eb-5ea654a397df None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.573601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fe36-e72d-41d8-b7eb-5ea654a397df None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.574097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fe36-e72d-41d8-b7eb-5ea654a397df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.596466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.598644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.604630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.605287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.606267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.607551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:46:53.607551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jul 18 10:46:53.607551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: Jul 18 10:46:53.617509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.633100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f5f3fe36-e72d-41d8-b7eb-5ea654a397df None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.635053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f5f3fe36-e72d-41d8-b7eb-5ea654a397df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.657100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 776/1548] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.658502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3fe36-e72d-41d8-b7eb-5ea654a397df None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.671906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3fe36-e72d-41d8-b7eb-5ea654a397df None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.685273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82052a11-ba77-4808-92a7-f620c8780d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.686419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82052a11-ba77-4808-92a7-f620c8780d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.686980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82052a11-ba77-4808-92a7-f620c8780d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.687485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82052a11-ba77-4808-92a7-f620c8780d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.688541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82052a11-ba77-4808-92a7-f620c8780d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:53.689122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82052a11-ba77-4808-92a7-f620c8780d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.691072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82052a11-ba77-4808-92a7-f620c8780d8d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.711835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 777/1549] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:53.717369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa81ae-a1ab-4240-9ab6-ffbd629f46b1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.717670 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa81ae-a1ab-4240-9ab6-ffbd629f46b1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.718238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa81ae-a1ab-4240-9ab6-ffbd629f46b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.733524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 773/1550] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:53.736536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.750648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4daa81ae-a1ab-4240-9ab6-ffbd629f46b1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.752402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4daa81ae-a1ab-4240-9ab6-ffbd629f46b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.771072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 778/1551] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.777942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa81ae-a1ab-4240-9ab6-ffbd629f46b1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.779610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322feba-b69e-4bc0-b883-2324441ed8af admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.793224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.794631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.795151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.812735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.814340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:53.815611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.820558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:46:53.820558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jul 18 10:46:53.820558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: Jul 18 10:46:53.821632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.823851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa81ae-a1ab-4240-9ab6-ffbd629f46b1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.836465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e822ad-c69a-468b-94e7-5aede20c5ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.840345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e822ad-c69a-468b-94e7-5aede20c5ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.841151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e822ad-c69a-468b-94e7-5aede20c5ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.841973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e822ad-c69a-468b-94e7-5aede20c5ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.844717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e822ad-c69a-468b-94e7-5aede20c5ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.847103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-19e822ad-c69a-468b-94e7-5aede20c5ac6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:46:53.847103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:46:53.847103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:46:53.849244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 774/1552] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:53.850310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.851458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e822ad-c69a-468b-94e7-5aede20c5ac6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.864560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcc149-b448-473c-843a-62a6fc8f7415 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.878474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcb115-28a5-4e27-99bd-dc874294c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.880257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcb115-28a5-4e27-99bd-dc874294c8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.882228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcb115-28a5-4e27-99bd-dc874294c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.882228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcb115-28a5-4e27-99bd-dc874294c8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.882228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcb115-28a5-4e27-99bd-dc874294c8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.883709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcb115-28a5-4e27-99bd-dc874294c8d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.903913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 775/1553] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:53.903913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8dc92-c374-435e-b15c-a21683e3ac7e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.903913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8dc92-c374-435e-b15c-a21683e3ac7e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.903913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8dc92-c374-435e-b15c-a21683e3ac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.924612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7c8dc92-c374-435e-b15c-a21683e3ac7e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:53.926640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7c8dc92-c374-435e-b15c-a21683e3ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:53.936383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 776/1554] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:53.937533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8dc92-c374-435e-b15c-a21683e3ac7e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:53.952772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8dc92-c374-435e-b15c-a21683e3ac7e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:53.959764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 779/1555] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:53.964404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657b8f-e866-4930-b28e-8f2710e7c616 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.964404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657b8f-e866-4930-b28e-8f2710e7c616 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.964404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657b8f-e866-4930-b28e-8f2710e7c616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.971474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52f7b4-ae23-459b-89ab-b6a8cab1d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:53.972618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f7b4-ae23-459b-89ab-b6a8cab1d89c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.973406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f7b4-ae23-459b-89ab-b6a8cab1d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:53.973648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f7b4-ae23-459b-89ab-b6a8cab1d89c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:53.974335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52f7b4-ae23-459b-89ab-b6a8cab1d89c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:53.974737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52f7b4-ae23-459b-89ab-b6a8cab1d89c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:53.976832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52f7b4-ae23-459b-89ab-b6a8cab1d89c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:53.990856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 777/1556] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:53.992406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261042ea-c6c8-41b9-9204-633e1578b909 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:53.992926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261042ea-c6c8-41b9-9204-633e1578b909 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.005771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261042ea-c6c8-41b9-9204-633e1578b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.006007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-02657b8f-e866-4930-b28e-8f2710e7c616 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.006007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-02657b8f-e866-4930-b28e-8f2710e7c616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.014503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-261042ea-c6c8-41b9-9204-633e1578b909 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.016185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-261042ea-c6c8-41b9-9204-633e1578b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.093945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 780/1557] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.095177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02657b8f-e866-4930-b28e-8f2710e7c616 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.108998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 778/1558] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.110520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261042ea-c6c8-41b9-9204-633e1578b909 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.122132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02657b8f-e866-4930-b28e-8f2710e7c616 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.123980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261042ea-c6c8-41b9-9204-633e1578b909 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.125318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:54.130338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246d2d-85ac-44b7-be06-fdeea9e64216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.131506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246d2d-85ac-44b7-be06-fdeea9e64216 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.136331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246d2d-85ac-44b7-be06-fdeea9e64216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.136331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246d2d-85ac-44b7-be06-fdeea9e64216 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.136551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eef35-8394-4d22-99d1-d60c8f32c240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.136551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eef35-8394-4d22-99d1-d60c8f32c240 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.136759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246d2d-85ac-44b7-be06-fdeea9e64216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.136886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eef35-8394-4d22-99d1-d60c8f32c240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.136886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eef35-8394-4d22-99d1-d60c8f32c240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.136886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eef35-8394-4d22-99d1-d60c8f32c240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.137645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246d2d-85ac-44b7-be06-fdeea9e64216 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.138667 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eef35-8394-4d22-99d1-d60c8f32c240 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.157697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 779/1559] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:54.159868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe990757-34fc-4967-969b-0eec43e8b117 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.160380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe990757-34fc-4967-969b-0eec43e8b117 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.161319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe990757-34fc-4967-969b-0eec43e8b117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.181672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fe990757-34fc-4967-969b-0eec43e8b117 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.182951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe990757-34fc-4967-969b-0eec43e8b117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.191380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 780/1560] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.193134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe990757-34fc-4967-969b-0eec43e8b117 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.203638 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe990757-34fc-4967-969b-0eec43e8b117 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.208824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.209533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.209836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.210113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.210955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.212997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.251666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 781/1561] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/projects => generated 393 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:54.253021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0903270-84e6-4b1c-be13-2a9c73af719e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.253317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0903270-84e6-4b1c-be13-2a9c73af719e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.253598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0903270-84e6-4b1c-be13-2a9c73af719e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.261934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:54.273565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c0903270-84e6-4b1c-be13-2a9c73af719e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.279708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c0903270-84e6-4b1c-be13-2a9c73af719e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.338569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 781/1562] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/users => generated 302 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:54.339477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.351398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316859cf-c749-42f2-83d9-c3e2410bc169 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.354850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 782/1563] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.356148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd406eae-6199-45de-a922-873e24000396 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.356435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd406eae-6199-45de-a922-873e24000396 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.356717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd406eae-6199-45de-a922-873e24000396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.364117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.365054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.365336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.365753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.366797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.368857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.376868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bd406eae-6199-45de-a922-873e24000396 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.378133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bd406eae-6199-45de-a922-873e24000396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.387652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 783/1564] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.389104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60433-c956-487f-a3df-3df8bb1ddb88 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.389401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60433-c956-487f-a3df-3df8bb1ddb88 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.389689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60433-c956-487f-a3df-3df8bb1ddb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.409425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-35d60433-c956-487f-a3df-3df8bb1ddb88 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.410713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35d60433-c956-487f-a3df-3df8bb1ddb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.419486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 784/1565] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.420325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60433-c956-487f-a3df-3df8bb1ddb88 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.432643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60433-c956-487f-a3df-3df8bb1ddb88 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.438199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.439123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.439409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.439688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] PATH_INFO: `/v3/projects/51d3c2992fa84e38880b885d48f220f6/users/8929cc243eca4a388bb134d21b0c2098/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.458703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d3c2992fa84e38880b885d48f220f6, user_id=8929cc243eca4a388bb134d21b0c2098, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.459922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:46:54.459922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:46:54.459922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:46:54.462211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.508073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 782/1566] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/projects => generated 403 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:54.508983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.588968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17b2c-8aad-499e-94f7-a58eafaa7872 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.596025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.597078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.598379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.598799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.600234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:54.600720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.603465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.615643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 785/1567] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:54 2022] PUT /identity/v3/projects/51d3c2992fa84e38880b885d48f220f6/users/8929cc243eca4a388bb134d21b0c2098/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:54.631582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 783/1568] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:54.631582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.631973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.640587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91642f31-3af9-47f7-bd1f-bc9a13206db0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.709511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6332e2-422c-4ecd-904c-fea0d552402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.710218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6332e2-422c-4ecd-904c-fea0d552402f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.710511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6332e2-422c-4ecd-904c-fea0d552402f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.710796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6332e2-422c-4ecd-904c-fea0d552402f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.711286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c01e5-cb47-469d-adcc-8c3bcfb9e488 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.711551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6332e2-422c-4ecd-904c-fea0d552402f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.713285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6332e2-422c-4ecd-904c-fea0d552402f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.716886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7204b2b-e773-4a24-9ff7-ae803d5be8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.717962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7204b2b-e773-4a24-9ff7-ae803d5be8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.718245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7204b2b-e773-4a24-9ff7-ae803d5be8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.718522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7204b2b-e773-4a24-9ff7-ae803d5be8e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.719354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7204b2b-e773-4a24-9ff7-ae803d5be8e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.721305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7204b2b-e773-4a24-9ff7-ae803d5be8e4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.726200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 784/1569] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:54.727632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ad1a-f5be-498b-8609-67ac4b6477bb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.727923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ad1a-f5be-498b-8609-67ac4b6477bb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.728203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ad1a-f5be-498b-8609-67ac4b6477bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.750505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d414ad1a-f5be-498b-8609-67ac4b6477bb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.751734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d414ad1a-f5be-498b-8609-67ac4b6477bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.760337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 785/1570] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.761360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ad1a-f5be-498b-8609-67ac4b6477bb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.771598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ad1a-f5be-498b-8609-67ac4b6477bb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.776353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.777050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.777338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.777615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.778356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:54.778657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.780280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.793077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 786/1571] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:54.794065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.807193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d131fd-5333-442c-9884-c7eccc8c848f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.812904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.813586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.813868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.814216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.814905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:54.815212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.817452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.830222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 787/1572] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:54.831208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.838710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:54.842523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3081390-2ed9-483a-9e7f-166a34ced79c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.847466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.848888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.849418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.849642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.850831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.852465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:54.863205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 786/1573] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:54.864409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c6e0-d091-41ec-aa5f-0fb4c814ebdf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.864706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c6e0-d091-41ec-aa5f-0fb4c814ebdf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.865073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c6e0-d091-41ec-aa5f-0fb4c814ebdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.893170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6724c6e0-d091-41ec-aa5f-0fb4c814ebdf None None] MFA Rules not processed for user `8929cc243eca4a388bb134d21b0c2098`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:54.894398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6724c6e0-d091-41ec-aa5f-0fb4c814ebdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:54.946450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:54.971498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 787/1574] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:54.972611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724c6e0-d091-41ec-aa5f-0fb4c814ebdf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:54.984795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724c6e0-d091-41ec-aa5f-0fb4c814ebdf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:54.989767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:54.990453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:54.990746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:54.991244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:54.991971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:54.993959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.010934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 788/1575] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/users => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:55.010934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.012449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5084-9cb8-4408-8d12-e9d6bedafc59 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.017699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.018385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.018674 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.018958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.020270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.022082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.033042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 789/1576] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:55.033947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.045219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60279cde-1cae-44cb-bf6c-c605c62d1f8d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.050131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.059194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.059194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.059194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.059194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:55.059194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.059194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.070546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 790/1577] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:55.071577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.082960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc795-c052-44c9-a9b9-67f437716914 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.088750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.089472 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.089756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.090032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.090765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.092739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.166160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 788/1578] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:54 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:55.167058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.236014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5f41-4353-4a87-b335-c6691311948d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.241043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.241739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.242080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.242375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.243274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.245258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.314696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 791/1579] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:55 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:55.315509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.389770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd16b4-e925-4eb7-90cb-9998a9b32227 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.394772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.396404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.396729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.397037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.397781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.399424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.410608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 792/1580] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:55.411823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.421817 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574ba83-e3d7-4671-aae9-d7ad6a1f8776 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.427283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.428016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.428266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.428494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.429221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.431155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.452253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 789/1581] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:55 2022] POST /identity/v3/projects => generated 377 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:55.456408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.528030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf478-be1d-4883-8386-28f106f86de1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.535592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.536309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.536708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.537039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.537826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.539986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.552740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 790/1582] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:55.553666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.566026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d619611-aa7f-4173-a634-02a37af1ee61 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.573268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.574275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.574568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.574953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] PATH_INFO: `/v3/projects/1e1c7c791e8749cea23f98bae6a1f88a/users/f371f9eb78c2416baa07b3331b64f097/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.594268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1c7c791e8749cea23f98bae6a1f88a, user_id=f371f9eb78c2416baa07b3331b64f097, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.597037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.634548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 793/1583] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:55 2022] POST /identity/v3/projects => generated 363 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:55.665921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.707619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1684e0-b0e7-491d-bf68-db19f96d31c3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.712493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.713285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.713504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.713795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.714514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.716516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.782208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 791/1584] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:55 2022] PUT /identity/v3/projects/1e1c7c791e8749cea23f98bae6a1f88a/users/f371f9eb78c2416baa07b3331b64f097/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:55.783900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.813358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:55.821247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:55.829579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 794/1585] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:55 2022] POST /identity/v3/projects => generated 375 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:55.830548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:55.855240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126e90c-c152-48e5-82c6-0dbd78bc818a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.898980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b272f39-d820-48ab-b924-f02f89cf5d3e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:55.903914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.904598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.904918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.905355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.906016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.908102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:55.919932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:55.920627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:55.920954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:55.921237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:55.921941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:55.923930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.033987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 795/1586] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:55 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:56.035630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.047068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3de922-fbff-46cb-b0df-1d3c613e2483 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.053037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.053717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.054004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.054282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.055015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.056646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.068382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 796/1587] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.069293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.070857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 792/1588] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:55 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:56.071956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.079820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e01d3a-b10e-45f5-af8a-3af707313c81 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.083379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe40a0-642b-4c77-babd-5d990fbfaf84 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.086032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91e91c-4674-4dda-98e0-8b71ef78fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.086754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91e91c-4674-4dda-98e0-8b71ef78fb8c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.087040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91e91c-4674-4dda-98e0-8b71ef78fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.087322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91e91c-4674-4dda-98e0-8b71ef78fb8c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.088025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91e91c-4674-4dda-98e0-8b71ef78fb8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.089140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.089814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.091007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91e91c-4674-4dda-98e0-8b71ef78fb8c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.091112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.091112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.091544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.093256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.105536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 793/1589] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.106368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.120816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccef05-e73d-4f89-b5c4-bdc72482b3ef admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.126400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.126400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.126400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.126400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.126908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.126908 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.237133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 797/1590] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:56 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:56.240626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d34ff-999b-4073-9018-2c029f9363fb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.241012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d34ff-999b-4073-9018-2c029f9363fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.241482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d34ff-999b-4073-9018-2c029f9363fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.246842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 794/1591] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:56 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:56.246842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.256555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c19ad-4898-415a-92d7-48c3ade16006 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.261930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.262642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.262931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.263213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.264024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.266115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.282305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e47d34ff-999b-4073-9018-2c029f9363fb None None] MFA Rules not processed for user `f371f9eb78c2416baa07b3331b64f097`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:56.283682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e47d34ff-999b-4073-9018-2c029f9363fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:56.368432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 795/1592] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:56 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:56.369478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.381066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e9a4-35b7-4fd7-88a8-45227bbce8a3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.383409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 798/1593] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:56.384426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d34ff-999b-4073-9018-2c029f9363fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.389740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.390851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.391327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.391792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.393079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.397068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.404513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d34ff-999b-4073-9018-2c029f9363fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.412953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.415947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 796/1594] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.416206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.416206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.416503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.416661 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.416838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.418831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.429289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c2c84-5c1d-4c7e-b00c-dc39964955df admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.434063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 799/1595] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.434612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.434792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.435535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.435829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.436109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] PATH_INFO: `/v3/projects/7184d0f3256344a2af18c639787e538f/users/e7a738986f87470681c6a578599a089f/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.448140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a9a3-1d17-44d2-a02a-cad22a30aca1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.453131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.454152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.454501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.454726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] PATH_INFO: `/v3/projects/9b3bf4aa5c674c38939bee4245cc57da/users/bd37fad46f9444e9a1e715c2be0ac13c/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.455558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7184d0f3256344a2af18c639787e538f, user_id=e7a738986f87470681c6a578599a089f, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.459082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.482602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3bf4aa5c674c38939bee4245cc57da, user_id=bd37fad46f9444e9a1e715c2be0ac13c, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.485941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.581734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 797/1596] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:56 2022] PUT /identity/v3/projects/7184d0f3256344a2af18c639787e538f/users/e7a738986f87470681c6a578599a089f/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:56.583316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.628061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 800/1597] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:56 2022] PUT /identity/v3/projects/9b3bf4aa5c674c38939bee4245cc57da/users/bd37fad46f9444e9a1e715c2be0ac13c/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:56.628852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:56.629350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.654115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ef4ba-57bf-4b20-8aae-cbd07a2f88c9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.660602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:56.711887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f9f04-6bef-45f1-aecf-8c6a365b3942 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.717082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.717996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.718117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.718367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.719140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.719595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.720946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.720946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.720946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.722448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.722570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.724737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.734965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 801/1598] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.738525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 798/1599] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.738765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.738857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.748073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e981d-0995-49d1-a520-f8da9c2f1fbf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.749838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26098cf-bdba-499c-bb5d-84bf6e3cb266 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.755431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.756711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.756711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.756711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.757508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.759355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.760512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.761605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.762090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.762555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.763620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.767299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.776798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 802/1600] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.778213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.791295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af84e6-8cd4-477e-bb7c-e52eae401882 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.796589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.797399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.802683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.802683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.802683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.805726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.819351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 803/1601] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.820280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.831468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf27fd-0206-46b7-9689-1df377efff2e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.837209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3a97-3465-4e3b-8c26-5ae1e75052a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.837891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3a97-3465-4e3b-8c26-5ae1e75052a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.838171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3a97-3465-4e3b-8c26-5ae1e75052a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.838462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3a97-3465-4e3b-8c26-5ae1e75052a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.839189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3a97-3465-4e3b-8c26-5ae1e75052a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.840863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3a97-3465-4e3b-8c26-5ae1e75052a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.852713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 804/1602] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.854101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d21397-a3c1-4dcd-a75c-cbc4f449528a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.854387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d21397-a3c1-4dcd-a75c-cbc4f449528a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.854653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d21397-a3c1-4dcd-a75c-cbc4f449528a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.872416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-76d21397-a3c1-4dcd-a75c-cbc4f449528a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:56.873644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-76d21397-a3c1-4dcd-a75c-cbc4f449528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:56.882317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 805/1603] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:56.883550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ca1d2-3203-40ed-a38d-66c9a5a1bdf6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.884060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ca1d2-3203-40ed-a38d-66c9a5a1bdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.884238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ca1d2-3203-40ed-a38d-66c9a5a1bdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.905695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 799/1604] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:56 2022] POST /identity/v3/projects => generated 405 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:56.905695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:56.914567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fe6ca1d2-3203-40ed-a38d-66c9a5a1bdf6 None None] MFA Rules not processed for user `bd37fad46f9444e9a1e715c2be0ac13c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:56.915790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe6ca1d2-3203-40ed-a38d-66c9a5a1bdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:56.975072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57e99a-f2db-4a00-b9e6-b0969e429770 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:56.980453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:56.981246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:56.981579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:56.981866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:56.982728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:56.984585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:56.996520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 800/1605] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:56.997534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.000450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 806/1606] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:57.004582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ca1d2-3203-40ed-a38d-66c9a5a1bdf6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.011675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffcbd3-ba4a-4b55-9ad6-db09b85a1f27 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.013305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ca1d2-3203-40ed-a38d-66c9a5a1bdf6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.016587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.017516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.017800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.018074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] PATH_INFO: `/v3/projects/1ccce20c1e66479191cb1f4ca2bc9f54/users/e43a7a71ae234b8aa65db59a1b2694ea/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.020730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.022021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.022543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.023049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.024297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.027610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.035702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccce20c1e66479191cb1f4ca2bc9f54, user_id=e43a7a71ae234b8aa65db59a1b2694ea, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.038466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.040592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 807/1607] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:57.041941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.058728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3a2c5-a34a-4aae-8614-556c9ade6139 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.067303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.068893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.069429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.069937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] PATH_INFO: `/v3/projects/04716d82e1084ec58b2a4909710a6ce8/users/3954fd4abf6b4c3a82280c762292a3e7/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.097592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04716d82e1084ec58b2a4909710a6ce8, user_id=3954fd4abf6b4c3a82280c762292a3e7, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.102867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.113707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 801/1608] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:56 2022] PUT /identity/v3/projects/1ccce20c1e66479191cb1f4ca2bc9f54/users/e43a7a71ae234b8aa65db59a1b2694ea/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:57.114654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.152836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:57.183039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b4d76-3dba-401f-ae79-3ad03725d8cd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.187782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.188687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.189224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.189504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] PATH_INFO: `/v3/projects/48497435674b41ccb34f6ff43026a232/users/7e032c8e5b024fa7a41cd1090e0f1ada/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.207591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48497435674b41ccb34f6ff43026a232, user_id=7e032c8e5b024fa7a41cd1090e0f1ada, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.211826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.245971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 808/1609] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:57 2022] PUT /identity/v3/projects/04716d82e1084ec58b2a4909710a6ce8/users/3954fd4abf6b4c3a82280c762292a3e7/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:57.246799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.249582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:57.282931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:57.311308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8fb48-4474-4c1b-bfba-75d2eaccce27 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.315354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.316021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.316314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.317499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.378713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 802/1610] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:57 2022] PUT /identity/v3/projects/48497435674b41ccb34f6ff43026a232/users/7e032c8e5b024fa7a41cd1090e0f1ada/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:57.381193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.399616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.401015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.412204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 809/1611] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:57.415441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.430192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d886c6-ed43-4d6d-a7a1-19716f619523 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.479958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4abd4-6f56-4014-9493-648f2ce43668 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.487222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.487919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.488829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.489200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.489907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.491890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.502315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.503203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.503483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.503755 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] PATH_INFO: `/v3/projects/6d288974aac54fa681cfafa262d019eb/users/ce15ddb712fa42dc8d487fe22d0959bd/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.526167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d288974aac54fa681cfafa262d019eb, user_id=ce15ddb712fa42dc8d487fe22d0959bd, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.531876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.588741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 803/1612] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:57 2022] POST /identity/v3/users => generated 309 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:57.589619 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.602212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728c100-d246-42b2-9c1a-a11461b73991 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.608329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.609238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.609521 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.609797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.610509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.612457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.669179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 810/1613] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:57 2022] PUT /identity/v3/projects/6d288974aac54fa681cfafa262d019eb/users/ce15ddb712fa42dc8d487fe22d0959bd/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:57.670442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.748599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14386b51-f919-4c76-86b4-c570c285a6e4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.754642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 804/1614] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:57 2022] POST /identity/v3/projects => generated 409 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:57.755504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.756560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d73260-855a-4ad9-b9b7-0768f3bc90ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.757441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73260-855a-4ad9-b9b7-0768f3bc90ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.757730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73260-855a-4ad9-b9b7-0768f3bc90ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.758013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73260-855a-4ad9-b9b7-0768f3bc90ec admin admin] PATH_INFO: `/v3/projects/f32dc706424247d0ac1dc7a6aa633107/users/3dbb489073d94425ad1a1b350768c428/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.773927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d73260-855a-4ad9-b9b7-0768f3bc90ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32dc706424247d0ac1dc7a6aa633107, user_id=3dbb489073d94425ad1a1b350768c428, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.776655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d73260-855a-4ad9-b9b7-0768f3bc90ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.783134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:57.845753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134f05f-19cd-4099-821d-c89b36072128 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.859808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 811/1615] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:57 2022] PUT /identity/v3/projects/f32dc706424247d0ac1dc7a6aa633107/users/3dbb489073d94425ad1a1b350768c428/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:57.861920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d524-f3c9-4985-b1b9-1a84d0ef5f13 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.862139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d524-f3c9-4985-b1b9-1a84d0ef5f13 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.862425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d524-f3c9-4985-b1b9-1a84d0ef5f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.865159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149a16f-57fc-42a7-9575-89b8eee1758f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.866020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149a16f-57fc-42a7-9575-89b8eee1758f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.866358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149a16f-57fc-42a7-9575-89b8eee1758f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.866646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149a16f-57fc-42a7-9575-89b8eee1758f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.868427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149a16f-57fc-42a7-9575-89b8eee1758f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.870375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149a16f-57fc-42a7-9575-89b8eee1758f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:57.883658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 805/1616] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:57.885354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa046da-bee2-43a1-acc4-83cdfd0834e6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.885793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa046da-bee2-43a1-acc4-83cdfd0834e6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.886173 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa046da-bee2-43a1-acc4-83cdfd0834e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.889575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9fc3d524-f3c9-4985-b1b9-1a84d0ef5f13 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:57.890905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9fc3d524-f3c9-4985-b1b9-1a84d0ef5f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:57.908574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3aa046da-bee2-43a1-acc4-83cdfd0834e6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:57.910002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3aa046da-bee2-43a1-acc4-83cdfd0834e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:57.927523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:57.974650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 812/1617] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:57.975562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3d524-f3c9-4985-b1b9-1a84d0ef5f13 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:57.984770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 806/1618] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:57.989312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77904a05-4d3f-49b7-bb71-8e95c378945f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.989312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77904a05-4d3f-49b7-bb71-8e95c378945f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.989312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77904a05-4d3f-49b7-bb71-8e95c378945f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.989597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3d524-f3c9-4985-b1b9-1a84d0ef5f13 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:57.994086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:57.994681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:57.995139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:57.995332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:57.996065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:57.997890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.005156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-77904a05-4d3f-49b7-bb71-8e95c378945f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:58.006382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-77904a05-4d3f-49b7-bb71-8e95c378945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:58.009494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 813/1619] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:58.010334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.016745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 807/1620] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:58.018241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7e832-2237-4c76-be97-4392c7ab7cc6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.018414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7e832-2237-4c76-be97-4392c7ab7cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.022098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7e832-2237-4c76-be97-4392c7ab7cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.023708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe813e36-55bc-4534-9d78-7973090ac2f2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.028766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.029701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.029992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.030274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.031255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.033313 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.042369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ffe7e832-2237-4c76-be97-4392c7ab7cc6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:58.043585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ffe7e832-2237-4c76-be97-4392c7ab7cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:58.051807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 808/1621] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:58.052758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7e832-2237-4c76-be97-4392c7ab7cc6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.062499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7e832-2237-4c76-be97-4392c7ab7cc6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.067328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.068311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.068674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.069093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] PATH_INFO: `/v3/projects/1ccce20c1e66479191cb1f4ca2bc9f54/users/e43a7a71ae234b8aa65db59a1b2694ea/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.071025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccce20c1e66479191cb1f4ca2bc9f54, user_id=e43a7a71ae234b8aa65db59a1b2694ea, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.073737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.089072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 809/1622] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:58 2022] PUT /identity/v3/projects/1ccce20c1e66479191cb1f4ca2bc9f54/users/e43a7a71ae234b8aa65db59a1b2694ea/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:58.089072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.173415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e1429-ea97-4d8e-82cb-90d8a15fe8b6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.182654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.182654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.182654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.182654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.183098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 814/1623] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:58 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:58.183098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.267205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3da6ee-36f5-467e-8bf1-e2932fb27593 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.271562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.272287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.272577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.272890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.275258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.276957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.284112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.286283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.288635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 810/1624] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:58.289518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.299270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055dce8-b1cd-4518-ad77-fe22ecef435e service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.303184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 815/1625] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:58.304160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.315740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b8366-4de5-47b9-9c83-675b56c346aa service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.318181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.319197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.320182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.320182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] PATH_INFO: `/v3/projects/aae0c10937244b1cb8a412c75159e3fd/users/427defbccc4548dfa42fc8fe5681d9e9/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.320458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.320583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.320984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.321333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.334955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.336793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.340457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae0c10937244b1cb8a412c75159e3fd, user_id=427defbccc4548dfa42fc8fe5681d9e9, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.343306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.350137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 816/1626] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:58.351632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.360878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb3785-8783-4f73-939a-70a434253ef6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.365898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fdaa7-7bc9-4d45-acdc-591aab3c70a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.366556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fdaa7-7bc9-4d45-acdc-591aab3c70a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.366837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fdaa7-7bc9-4d45-acdc-591aab3c70a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.367204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fdaa7-7bc9-4d45-acdc-591aab3c70a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.378129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fdaa7-7bc9-4d45-acdc-591aab3c70a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.380377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fdaa7-7bc9-4d45-acdc-591aab3c70a3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.392493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 817/1627] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:58.395882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487a15d-647e-410a-a647-07f4ce3294b2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.395882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487a15d-647e-410a-a647-07f4ce3294b2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.395882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487a15d-647e-410a-a647-07f4ce3294b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.432863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3487a15d-647e-410a-a647-07f4ce3294b2 None None] MFA Rules not processed for user `e43a7a71ae234b8aa65db59a1b2694ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:58.434069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3487a15d-647e-410a-a647-07f4ce3294b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:58.453508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 811/1628] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:58 2022] PUT /identity/v3/projects/aae0c10937244b1cb8a412c75159e3fd/users/427defbccc4548dfa42fc8fe5681d9e9/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:58.453508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.464606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:58.487886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:58.557374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 818/1629] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:58 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:58.558378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487a15d-647e-410a-a647-07f4ce3294b2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.571412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bee2ed-4da7-4de4-bab2-80284726db72 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.573178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487a15d-647e-410a-a647-07f4ce3294b2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.577815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.581676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.581676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.581676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.581676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.582189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.582189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.582189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.582189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.582743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.583175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.584299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.595726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 819/1630] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:58.596367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 812/1631] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:58.596720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.603145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.623247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13674bef-c202-405f-8750-4a5f9f3c870f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.628690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.630967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.631349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.631629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] PATH_INFO: `/v3/projects/aae0c10937244b1cb8a412c75159e3fd/users/427defbccc4548dfa42fc8fe5681d9e9/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.634122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae0c10937244b1cb8a412c75159e3fd, user_id=427defbccc4548dfa42fc8fe5681d9e9, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.637033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.651635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 813/1632] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:58 2022] PUT /identity/v3/projects/aae0c10937244b1cb8a412c75159e3fd/users/427defbccc4548dfa42fc8fe5681d9e9/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:58.656275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:58.703039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507548c-2fe8-4730-bb94-5e538ace872a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.795877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec94d5d-a266-411d-894c-2c93ced106e3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:58.809794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.811733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.812025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.812316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] PATH_INFO: `/v3/projects/1891541b61be4afbae5eb25c9f718e31/users/850863d663204ef2a7be54e1e391696d/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.829645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c3e81-1cc9-4ea1-b7dd-ccac0229095a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:58.830145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3e81-1cc9-4ea1-b7dd-ccac0229095a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.830436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3e81-1cc9-4ea1-b7dd-ccac0229095a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.830710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3e81-1cc9-4ea1-b7dd-ccac0229095a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.839312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1891541b61be4afbae5eb25c9f718e31, user_id=850863d663204ef2a7be54e1e391696d, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.842722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.926429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c3e81-1cc9-4ea1-b7dd-ccac0229095a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:58.928089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c3e81-1cc9-4ea1-b7dd-ccac0229095a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:58.940116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 820/1633] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:58.942614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246efb5-c3b0-4fa1-a511-e2ff9783a4a2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:58.943483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246efb5-c3b0-4fa1-a511-e2ff9783a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:58.944140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246efb5-c3b0-4fa1-a511-e2ff9783a4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:58.959616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 814/1634] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:58 2022] PUT /identity/v3/projects/1891541b61be4afbae5eb25c9f718e31/users/850863d663204ef2a7be54e1e391696d/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:58.961105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:59.005651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:59.016926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4246efb5-c3b0-4fa1-a511-e2ff9783a4a2 None None] MFA Rules not processed for user `427defbccc4548dfa42fc8fe5681d9e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:59.018179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4246efb5-c3b0-4fa1-a511-e2ff9783a4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:59.082846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d897482-5332-4a31-aed0-c37f04eefefb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:59.093294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:59.094155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:59.094454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:59.469824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:59.469824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:59.469824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:59.469824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 815/1635] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:59.469824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:59.469824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33044b63-680f-479b-b6cd-94ff5c2e3ffa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:59.474249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fb5ff-c06c-4410-a136-ecbd74f1a0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:59.474249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb5ff-c06c-4410-a136-ecbd74f1a0cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:59.474249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb5ff-c06c-4410-a136-ecbd74f1a0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:59.474249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb5ff-c06c-4410-a136-ecbd74f1a0cd admin admin] PATH_INFO: `/v3/projects/1891541b61be4afbae5eb25c9f718e31/users/850863d663204ef2a7be54e1e391696d/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:59.474990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fb5ff-c06c-4410-a136-ecbd74f1a0cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1891541b61be4afbae5eb25c9f718e31, user_id=850863d663204ef2a7be54e1e391696d, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:59.474990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fb5ff-c06c-4410-a136-ecbd74f1a0cd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:59.474990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 816/1636] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:59 2022] PUT /identity/v3/projects/1891541b61be4afbae5eb25c9f718e31/users/850863d663204ef2a7be54e1e391696d/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:59.475513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 821/1637] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:59.476273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53136-ab51-4442-9408-34536519e383 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:59.476273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53136-ab51-4442-9408-34536519e383 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:59.476685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246efb5-c3b0-4fa1-a511-e2ff9783a4a2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:59.477068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53136-ab51-4442-9408-34536519e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:59.477068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:59.477068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5fb53136-ab51-4442-9408-34536519e383 None None] MFA Rules not processed for user `850863d663204ef2a7be54e1e391696d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:59.477068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5fb53136-ab51-4442-9408-34536519e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:59.478243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246efb5-c3b0-4fa1-a511-e2ff9783a4a2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:59.478783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 817/1638] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:46:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:59.478783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb53136-ab51-4442-9408-34536519e383 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:59.479421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:59.479421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:59.479421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:59.479421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:59.479841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb53136-ab51-4442-9408-34536519e383 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:59.479841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:59.479841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:59.479841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:59.480276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:59.480403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:59.480403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:59.480403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 822/1639] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:59 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:46:59.501535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:59.503060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:59.515295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 818/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:59 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:59.571715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:59.580312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:59.592949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cc8e-8c49-49f5-abb1-56100ba21ff7 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:59.602479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:59.603889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:59.604493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:59.605231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:59.627916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:59.630086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:59.646311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 823/1641] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:00.357660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:00.369707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f08716-de40-4f8c-90dc-53d72038dfa8 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:00.384463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7ff3f-7a0f-4064-91f8-b0390e59be35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:00.385341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7ff3f-7a0f-4064-91f8-b0390e59be35 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:00.385784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7ff3f-7a0f-4064-91f8-b0390e59be35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:00.386081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7ff3f-7a0f-4064-91f8-b0390e59be35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:00.506043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:00.515771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:00.572388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7ff3f-7a0f-4064-91f8-b0390e59be35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:00.578916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7ff3f-7a0f-4064-91f8-b0390e59be35 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:00.618832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 819/1642] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:00.679170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f9359-7b36-454c-9e1e-ecbfd3af2469 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:00.699139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:00.700653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:00.701203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:00.701700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] PATH_INFO: `/v3/users/f371f9eb78c2416baa07b3331b64f097` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:00.709789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f371f9eb78c2416baa07b3331b64f097)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:00.713564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:01.069986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f371f9eb78c2416baa07b3331b64f097'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:01.089081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f371f9eb78c2416baa07b3331b64f097'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:01.222725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f371f9eb78c2416baa07b3331b64f097'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:01.225458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f371f9eb78c2416baa07b3331b64f097'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:01.228597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f371f9eb78c2416baa07b3331b64f097'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:01.261328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 824/1643] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:00 2022] DELETE /identity/v3/users/f371f9eb78c2416baa07b3331b64f097 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:01.266034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:01.266417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:01.266777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:01.288203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:01.289906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:01.406110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 820/1644] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:01.419946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:01.422836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:01.570189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d148bf-8373-44e3-9eb5-a91e9076db80 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:01.575273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:01.576116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:01.576467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:01.576958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:01.587989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:01.589777 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:01.605014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 825/1645] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:03.099617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:03.116020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:03.152177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6864df-5253-459b-a151-ee48e0188e52 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:03.175132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:03.178439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:03.182048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:03.182887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] PATH_INFO: `/v3/projects/1e1c7c791e8749cea23f98bae6a1f88a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:03.185465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1c7c791e8749cea23f98bae6a1f88a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:03.200553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:03.446328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1c7c791e8749cea23f98bae6a1f88a', 'request_id': 'req-25d0a7c3-4f0f-4640-89e1-205421652fe1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:03.458218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1c7c791e8749cea23f98bae6a1f88a', 'request_id': 'req-25d0a7c3-4f0f-4640-89e1-205421652fe1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:03.752951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1c7c791e8749cea23f98bae6a1f88a', 'request_id': 'req-25d0a7c3-4f0f-4640-89e1-205421652fe1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:03.778998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] The token cache is being invalidate because project 1e1c7c791e8749cea23f98bae6a1f88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:03.779285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:03.781215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:03.783171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 821/1646] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:03 2022] DELETE /identity/v3/projects/1e1c7c791e8749cea23f98bae6a1f88a => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:03.787571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:03.798670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a817d8-aabf-499b-9024-00e053822b6a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:03.829465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:04.067884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:04.069865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:04.071352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:04.074386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] PATH_INFO: `/v3/users/427defbccc4548dfa42fc8fe5681d9e9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:04.080310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=427defbccc4548dfa42fc8fe5681d9e9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:04.094815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:04.452743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '427defbccc4548dfa42fc8fe5681d9e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:04.491180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '427defbccc4548dfa42fc8fe5681d9e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:04.610844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '427defbccc4548dfa42fc8fe5681d9e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:04.614811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '427defbccc4548dfa42fc8fe5681d9e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:04.614811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-85cd639f-f42f-4db7-afd5-c12ead7b9c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '427defbccc4548dfa42fc8fe5681d9e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:04.640540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 826/1647] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:03 2022] DELETE /identity/v3/users/427defbccc4548dfa42fc8fe5681d9e9 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:06.688512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:06.690669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:06.718552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a7c3-4f0f-4640-89e1-205421652fe1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:06.890473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:06.891727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.892239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.892829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] PATH_INFO: `/v3/projects/aae0c10937244b1cb8a412c75159e3fd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.895093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae0c10937244b1cb8a412c75159e3fd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:06.899104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:06.914165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:07.099616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae0c10937244b1cb8a412c75159e3fd', 'request_id': 'req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:07.132987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae0c10937244b1cb8a412c75159e3fd', 'request_id': 'req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:07.284202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae0c10937244b1cb8a412c75159e3fd', 'request_id': 'req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:07.326090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] The token cache is being invalidate because project aae0c10937244b1cb8a412c75159e3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:07.330744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:07.330744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6cd3c481-ccaf-4e10-90a2-52ddee52f8a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:07.330744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 822/1648] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:06 2022] DELETE /identity/v3/projects/aae0c10937244b1cb8a412c75159e3fd => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:07.338528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b9e1-04b7-4c63-a890-4e79bfba26b7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:07.339074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b9e1-04b7-4c63-a890-4e79bfba26b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:07.339160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b9e1-04b7-4c63-a890-4e79bfba26b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:07.368371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fa51b9e1-04b7-4c63-a890-4e79bfba26b7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:07.372818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa51b9e1-04b7-4c63-a890-4e79bfba26b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:07.484447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 827/1649] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:07.493159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fc099-ba40-45de-a9b2-5a108b8e997a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:07.493561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fc099-ba40-45de-a9b2-5a108b8e997a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:07.495364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fc099-ba40-45de-a9b2-5a108b8e997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:07.520205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e98fc099-ba40-45de-a9b2-5a108b8e997a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:07.521779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e98fc099-ba40-45de-a9b2-5a108b8e997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:07.562770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 823/1650] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:07.565463 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51b9e1-04b7-4c63-a890-4e79bfba26b7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:07.585700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51b9e1-04b7-4c63-a890-4e79bfba26b7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:07.598056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:07.603031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:07.606696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:07.607246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:07.609139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:47:07.609738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:07.613591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:07.640106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 828/1651] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:47:07.644343 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fc099-ba40-45de-a9b2-5a108b8e997a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:07.661996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fc099-ba40-45de-a9b2-5a108b8e997a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:07.668991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:07.671852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:07.672315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:07.672671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:07.673753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:07.679178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:07.856144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 824/1652] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:07 2022] POST /identity/v3/projects => generated 403 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:47:07.857060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:07.870211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301faa2a-3758-403d-8bcd-b3a35c570ca1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:07.898538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:08.028107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:08.030263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:08.031169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:08.031466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:08.034404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:08.039034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:08.164589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 829/1653] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:07 2022] POST /identity/v3/users => generated 302 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:08.171666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:08.217803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d24c-6e02-4700-8123-d95b195f768e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:08.243445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:08.245178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:08.253693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:08.260387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:08.261889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:08.272880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:08.354617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:08.370355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 825/1654] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:08.377308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b8065-18a9-4d9c-ae60-1f9759503b31 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:08.411925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad3bf1-9282-4c54-97ea-86d720a8c5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:08.413065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad3bf1-9282-4c54-97ea-86d720a8c5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:08.413534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad3bf1-9282-4c54-97ea-86d720a8c5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:08.413974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad3bf1-9282-4c54-97ea-86d720a8c5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:08.415040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad3bf1-9282-4c54-97ea-86d720a8c5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:08.422826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad3bf1-9282-4c54-97ea-86d720a8c5b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:08.495768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:08.532425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96265-20b5-49e5-9a98-2280016103b5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:08.533332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 830/1655] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:08.555016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:08.559169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:08.559169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:08.559169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] PATH_INFO: `/v3/projects/1933792aea1c42a9b766cfd51820c7e1/users/0b1882bcbe8543f18d617158f582aeb6/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:08.601663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1933792aea1c42a9b766cfd51820c7e1, user_id=0b1882bcbe8543f18d617158f582aeb6, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:08.607755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:08.737505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93a96c-e50e-4a8e-9590-4032ea985730 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:08.746411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93a96c-e50e-4a8e-9590-4032ea985730 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:08.746951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93a96c-e50e-4a8e-9590-4032ea985730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:08.758077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 826/1656] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:08 2022] PUT /identity/v3/projects/1933792aea1c42a9b766cfd51820c7e1/users/0b1882bcbe8543f18d617158f582aeb6/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:08.793332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f93a96c-e50e-4a8e-9590-4032ea985730 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:08.794964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f93a96c-e50e-4a8e-9590-4032ea985730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:08.944520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 831/1657] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:09.013595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:09.016970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:09.209393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4bded-492f-4a7b-ace3-d5ef455a3690 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:09.230610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d4105-8789-4f62-add0-791b65f5d85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:09.233982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d4105-8789-4f62-add0-791b65f5d85d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:09.234555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d4105-8789-4f62-add0-791b65f5d85d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:09.235103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d4105-8789-4f62-add0-791b65f5d85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:09.270646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d4105-8789-4f62-add0-791b65f5d85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:09.275150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d4105-8789-4f62-add0-791b65f5d85d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:09.319390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 827/1658] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:15.977439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5346a-c298-42bc-b9a1-389a29cad6b6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:15.977956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5346a-c298-42bc-b9a1-389a29cad6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:15.978113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5346a-c298-42bc-b9a1-389a29cad6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:16.041507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-96d5346a-c298-42bc-b9a1-389a29cad6b6 None None] MFA Rules not processed for user `e7a738986f87470681c6a578599a089f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:16.044010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-96d5346a-c298-42bc-b9a1-389a29cad6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:16.260874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 832/1659] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:16.264998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e61fd2-b2a0-47d7-8b19-dc3f7eaf85dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:16.265509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e61fd2-b2a0-47d7-8b19-dc3f7eaf85dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:16.267205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e61fd2-b2a0-47d7-8b19-dc3f7eaf85dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:16.368623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-46e61fd2-b2a0-47d7-8b19-dc3f7eaf85dd None None] MFA Rules not processed for user `e7a738986f87470681c6a578599a089f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:16.370750 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46e61fd2-b2a0-47d7-8b19-dc3f7eaf85dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:16.400975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 828/1660] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:16.460133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f772-4387-4446-85a0-20e5dc50aa33 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:16.460468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f772-4387-4446-85a0-20e5dc50aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:16.460582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f772-4387-4446-85a0-20e5dc50aa33 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:16.462187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 833/1661] 10.222.0.49 () {54 vars in 962 bytes} [Mon Jul 18 10:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:16.502919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f4d5-9177-4dc8-8b5a-5af9371472a6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:16.507544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f4d5-9177-4dc8-8b5a-5af9371472a6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:16.507544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f4d5-9177-4dc8-8b5a-5af9371472a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:16.633987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f772-4387-4446-85a0-20e5dc50aa33 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:16.636274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a93f772-4387-4446-85a0-20e5dc50aa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:16.644318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1f30f4d5-9177-4dc8-8b5a-5af9371472a6 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:16.650012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1f30f4d5-9177-4dc8-8b5a-5af9371472a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:16.687104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f772-4387-4446-85a0-20e5dc50aa33 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:16.705921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:16.706706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:16.707932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:16.709669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:16.709891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] PATH_INFO: `/v3/users/bd37fad46f9444e9a1e715c2be0ac13c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:16.714396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd37fad46f9444e9a1e715c2be0ac13c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:16.721245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:16.817017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:16.851800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 829/1662] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Jul 18 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:16.854625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f1bd1-acb5-4f73-85af-e86f14207db8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:16.856086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f1bd1-acb5-4f73-85af-e86f14207db8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:16.856585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f1bd1-acb5-4f73-85af-e86f14207db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:16.915292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-346f1bd1-acb5-4f73-85af-e86f14207db8 None None] MFA Rules not processed for user `0b1882bcbe8543f18d617158f582aeb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:16.917163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-346f1bd1-acb5-4f73-85af-e86f14207db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:17.059191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd37fad46f9444e9a1e715c2be0ac13c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:17.122489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd37fad46f9444e9a1e715c2be0ac13c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:17.126131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 830/1663] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:17.127070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f1bd1-acb5-4f73-85af-e86f14207db8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:17.158852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f1bd1-acb5-4f73-85af-e86f14207db8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:17.173919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:17.175011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:17.176286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:17.176766 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:17.187411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:17.189338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:17.205838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 831/1664] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:17.207799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:17.220049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8526289-bb4c-4bb8-bb2a-96003e4800a5 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:17.226992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:17.228127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:17.228560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:17.231042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd37fad46f9444e9a1e715c2be0ac13c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:17.232054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd37fad46f9444e9a1e715c2be0ac13c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:17.233027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00ae3ce1-5b89-4b66-81bd-e1e72bd11c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd37fad46f9444e9a1e715c2be0ac13c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:17.241129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:17.242384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:17.253910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:17.286974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 834/1665] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:16 2022] DELETE /identity/v3/users/bd37fad46f9444e9a1e715c2be0ac13c => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:17.296355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecdc6d-de05-4cb6-9183-648816a5b19f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:17.296664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecdc6d-de05-4cb6-9183-648816a5b19f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:17.297220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecdc6d-de05-4cb6-9183-648816a5b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:17.355669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-66ecdc6d-de05-4cb6-9183-648816a5b19f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:17.356750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66ecdc6d-de05-4cb6-9183-648816a5b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:17.464364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 832/1666] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:17 2022] POST /identity/v3/projects => generated 403 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:17.466629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:17.478670 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:17.604223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 835/1667] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:47:17.704441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71822bd6-4d71-4d1c-8361-eef1a7074230 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:17.727056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:17.728078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:17.728570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:17.733158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:17.737455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:17.747498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:17.768192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5c488-cba8-4c4c-a97b-f1c4151a9bfb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:17.768192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5c488-cba8-4c4c-a97b-f1c4151a9bfb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:17.769611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5c488-cba8-4c4c-a97b-f1c4151a9bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:17.882364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f6b5c488-cba8-4c4c-a97b-f1c4151a9bfb None None] MFA Rules not processed for user `3dbb489073d94425ad1a1b350768c428`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:17.888769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f6b5c488-cba8-4c4c-a97b-f1c4151a9bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:17.890234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 833/1668] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:17 2022] POST /identity/v3/users => generated 308 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:17.903897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:18.015470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158cefb-b738-454b-b75d-c17f9195b6ed admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:18.066282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:18.067553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.068073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.073174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.074077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:18.075844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:18.107823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 834/1669] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:18.108795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:18.116685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:18.172026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 836/1670] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:18.194012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010ab48-6bc1-4a5f-b0c0-9dfbc1d38894 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.197870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010ab48-6bc1-4a5f-b0c0-9dfbc1d38894 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.199723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010ab48-6bc1-4a5f-b0c0-9dfbc1d38894 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.204676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 837/1671] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 18 10:47:18 2022] GET /identity => generated 270 bytes in 32 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:18.205884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010ab48-6bc1-4a5f-b0c0-9dfbc1d38894 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:18.226455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010ab48-6bc1-4a5f-b0c0-9dfbc1d38894 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:18.247726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9159e42-aca8-4538-8b30-fb2c73bff4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:18.249569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9159e42-aca8-4538-8b30-fb2c73bff4a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.249749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9159e42-aca8-4538-8b30-fb2c73bff4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.257372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9159e42-aca8-4538-8b30-fb2c73bff4a6 admin admin] PATH_INFO: `/v3/projects/295a4c37c02d4c0ab7fdab38198bb254/users/c8d33eb0f05f41ca80c2c3b7646a0597/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.341163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9159e42-aca8-4538-8b30-fb2c73bff4a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295a4c37c02d4c0ab7fdab38198bb254, user_id=c8d33eb0f05f41ca80c2c3b7646a0597, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:18.344576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9159e42-aca8-4538-8b30-fb2c73bff4a6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:18.347924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537256ef-7868-4ab4-8170-91fdf8c5478d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:18.371343 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef86a9-e833-4ca4-8e25-7e226fcc22cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:18.372836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef86a9-e833-4ca4-8e25-7e226fcc22cc service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.373570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef86a9-e833-4ca4-8e25-7e226fcc22cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.376639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef86a9-e833-4ca4-8e25-7e226fcc22cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.411978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef86a9-e833-4ca4-8e25-7e226fcc22cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:18.417320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef86a9-e833-4ca4-8e25-7e226fcc22cc service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:18.449117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 835/1672] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:18.450840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665325e-e6a0-48b3-b81f-e0ed5c2a06c5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.453753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665325e-e6a0-48b3-b81f-e0ed5c2a06c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.453753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665325e-e6a0-48b3-b81f-e0ed5c2a06c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.499660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5665325e-e6a0-48b3-b81f-e0ed5c2a06c5 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:18.500999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5665325e-e6a0-48b3-b81f-e0ed5c2a06c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:18.598506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 838/1673] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:18 2022] PUT /identity/v3/projects/295a4c37c02d4c0ab7fdab38198bb254/users/c8d33eb0f05f41ca80c2c3b7646a0597/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:18.602184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2121-7e0e-4e9f-8216-bfc259cf22f2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.602471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2121-7e0e-4e9f-8216-bfc259cf22f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.602778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2121-7e0e-4e9f-8216-bfc259cf22f2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.610462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 839/1674] 10.222.0.49 () {54 vars in 962 bytes} [Mon Jul 18 10:47:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:18.611402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2121-7e0e-4e9f-8216-bfc259cf22f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:18.693217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 836/1675] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:47:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:18.707103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb1907-e910-4a9c-b19a-cbff42f45941 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.707535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb1907-e910-4a9c-b19a-cbff42f45941 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.707743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb1907-e910-4a9c-b19a-cbff42f45941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.755495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4adb1907-e910-4a9c-b19a-cbff42f45941 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:18.756821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4adb1907-e910-4a9c-b19a-cbff42f45941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:18.795709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2121-7e0e-4e9f-8216-bfc259cf22f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:18.810042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:18.810042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:18.810042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:18.810042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:18.810435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:18.817938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:18.872408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 840/1676] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:18.874521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:18.878133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:19.069598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 837/1677] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Jul 18 10:47:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:19.071856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fd8a2-be06-4111-ac2f-65f97a7bf175 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:19.072170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fd8a2-be06-4111-ac2f-65f97a7bf175 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:19.072456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fd8a2-be06-4111-ac2f-65f97a7bf175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:19.152800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403b9a2-e65d-4b51-8135-aedafd7cd6e6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:19.166233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:19.166920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:19.167204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:19.167475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:19.191672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:19.195385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:19.205771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e20fd8a2-be06-4111-ac2f-65f97a7bf175 None None] MFA Rules not processed for user `7e032c8e5b024fa7a41cd1090e0f1ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:19.207722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e20fd8a2-be06-4111-ac2f-65f97a7bf175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:19.221771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:19.244677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 841/1678] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:19.245898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:19.294946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8098b-5b20-4251-9929-3d18376cc883 service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:19.312989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:19.312989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:19.315421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:19.317030 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] PATH_INFO: `/v3/projects/295a4c37c02d4c0ab7fdab38198bb254/users/c8d33eb0f05f41ca80c2c3b7646a0597/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:19.329010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295a4c37c02d4c0ab7fdab38198bb254, user_id=c8d33eb0f05f41ca80c2c3b7646a0597, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:19.333522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:19.382220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 842/1679] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:19 2022] PUT /identity/v3/projects/295a4c37c02d4c0ab7fdab38198bb254/users/c8d33eb0f05f41ca80c2c3b7646a0597/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:19.383387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:19.463354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 838/1680] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:19.465312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:19.465823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:19.471621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:19.566987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] MFA Rules not processed for user `3954fd4abf6b4c3a82280c762292a3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:19.570686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:19.633306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73750e-46e5-4555-b293-eb9f6e930445 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:19.646929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:19.647621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:19.647910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:19.648984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:19.819483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 839/1681] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:19.820730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:19.834100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:19.861912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:19.863767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:19.897081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 843/1682] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:19.898036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:19.901218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:20.023792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d1390-3f5f-4647-8c05-67de01a14bf3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:20.041686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:20.044499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:20.054808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:20.055512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] PATH_INFO: `/v3/projects/9b3bf4aa5c674c38939bee4245cc57da` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:20.058064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b3bf4aa5c674c38939bee4245cc57da)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:20.061349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:20.160327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500b976-50aa-4d7b-b065-22cca6ee706e service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:20.172236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:20.195539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:20.197137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:20.197679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:20.198200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] PATH_INFO: `/v3/users/8929cc243eca4a388bb134d21b0c2098` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:20.200676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8929cc243eca4a388bb134d21b0c2098)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:20.204613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:20.369177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b3bf4aa5c674c38939bee4245cc57da', 'request_id': 'req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.415241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b3bf4aa5c674c38939bee4245cc57da', 'request_id': 'req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.521131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8929cc243eca4a388bb134d21b0c2098'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.543314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b3bf4aa5c674c38939bee4245cc57da', 'request_id': 'req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.557662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8929cc243eca4a388bb134d21b0c2098'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.580358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] The token cache is being invalidate because project 9b3bf4aa5c674c38939bee4245cc57da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:20.580912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.584137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-16f8a2ff-43c8-43b2-b6ba-2e5eaa60cf9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.590611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 840/1683] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:19 2022] DELETE /identity/v3/projects/9b3bf4aa5c674c38939bee4245cc57da => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:20.593797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898e8df-a4d6-425d-85be-295065273900 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:20.594320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898e8df-a4d6-425d-85be-295065273900 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:20.596279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898e8df-a4d6-425d-85be-295065273900 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:20.605890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 841/1684] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 18 10:47:20 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:20.608017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898e8df-a4d6-425d-85be-295065273900 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:20.618556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0898e8df-a4d6-425d-85be-295065273900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:20.697336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8929cc243eca4a388bb134d21b0c2098'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.699185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8929cc243eca4a388bb134d21b0c2098'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.701018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64b139bf-74dc-415b-9896-418a7b7c3ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8929cc243eca4a388bb134d21b0c2098'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:20.747657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 844/1685] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:19 2022] DELETE /identity/v3/users/8929cc243eca4a388bb134d21b0c2098 => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:20.750998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca185-f614-4401-a1d7-436dc33f0e80 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:20.751587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca185-f614-4401-a1d7-436dc33f0e80 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:20.754349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca185-f614-4401-a1d7-436dc33f0e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:20.895975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-984ca185-f614-4401-a1d7-436dc33f0e80 None None] MFA Rules not processed for user `a48b0fdacb2c43649916304e1237f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:20.900183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-984ca185-f614-4401-a1d7-436dc33f0e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:20.907307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898e8df-a4d6-425d-85be-295065273900 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:21.110897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 845/1686] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:21.120989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:21.121420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:21.121809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:21.166712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:21.173002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:21.173002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:21.173002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] PATH_INFO: `/v3/users/850863d663204ef2a7be54e1e391696d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:21.173462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:21.174466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=850863d663204ef2a7be54e1e391696d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:21.176615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:21.178959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:47:21.178959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 18 10:47:21.178959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: Jul 18 10:47:21.185252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:21.208435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 846/1687] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:21.213217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:21.220678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:21.486532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89117e40-b8ea-41b5-85f3-0cb5a92f3f92 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:21.517617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:21.518843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:21.519509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:21.519946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:21.527615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:21.661049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850863d663204ef2a7be54e1e391696d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:21.707985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850863d663204ef2a7be54e1e391696d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:21.715769 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:21.776285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850863d663204ef2a7be54e1e391696d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:21.823567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:21.828005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:21.877613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 847/1688] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:21.879644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:21.881958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:21.912579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb3727-107d-44f0-a54d-220dba0f360f service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:21.919261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aabd3-08fc-4565-8926-1a21af6bb65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:21.920378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aabd3-08fc-4565-8926-1a21af6bb65c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:21.920908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aabd3-08fc-4565-8926-1a21af6bb65c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:21.921741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aabd3-08fc-4565-8926-1a21af6bb65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:21.946573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aabd3-08fc-4565-8926-1a21af6bb65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:21.966940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aabd3-08fc-4565-8926-1a21af6bb65c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:21.987495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850863d663204ef2a7be54e1e391696d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:21.988183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850863d663204ef2a7be54e1e391696d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:21.998820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 842/1689] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:20 2022] DELETE /identity/v3/users/850863d663204ef2a7be54e1e391696d => generated 0 bytes in 1391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:22.001745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:22.010460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:22.024136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 848/1690] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:22.032962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:22.032962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:22.033265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:22.160282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] MFA Rules not processed for user `ce15ddb712fa42dc8d487fe22d0959bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:22.178894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:22.321636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d7ef2-78f8-4f2a-a1a2-02e76fdd70c4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:22.338659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb70a0-0da1-4955-b8cf-08eef5fbc489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:22.342613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb70a0-0da1-4955-b8cf-08eef5fbc489 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:22.343808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb70a0-0da1-4955-b8cf-08eef5fbc489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:22.344379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb70a0-0da1-4955-b8cf-08eef5fbc489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:22.350097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24bb70a0-0da1-4955-b8cf-08eef5fbc489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:22.481057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 849/1691] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:22.482111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:22.483634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:22.525501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb70a0-0da1-4955-b8cf-08eef5fbc489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:22.527505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb70a0-0da1-4955-b8cf-08eef5fbc489 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:22.558684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 843/1692] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:22.561145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf4a7-1e98-4f72-8bb1-167bd83eb317 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:22.561383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf4a7-1e98-4f72-8bb1-167bd83eb317 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:22.561665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf4a7-1e98-4f72-8bb1-167bd83eb317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:22.591782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b64cf4a7-1e98-4f72-8bb1-167bd83eb317 None None] MFA Rules not processed for user `ce15ddb712fa42dc8d487fe22d0959bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:22.593432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b64cf4a7-1e98-4f72-8bb1-167bd83eb317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:22.609794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 844/1693] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:22.694693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f4bf6-a6ef-497d-aa2d-3d2ca7d842c1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:22.710049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4fc37-d38d-4c3c-89d1-1880f6bc1905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:22.710769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4fc37-d38d-4c3c-89d1-1880f6bc1905 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:22.711330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4fc37-d38d-4c3c-89d1-1880f6bc1905 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:22.714845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4fc37-d38d-4c3c-89d1-1880f6bc1905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:22.719504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-abe4fc37-d38d-4c3c-89d1-1880f6bc1905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:22.839869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:22.873281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4fc37-d38d-4c3c-89d1-1880f6bc1905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:22.874961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4fc37-d38d-4c3c-89d1-1880f6bc1905 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:22.908260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 850/1694] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:23.108050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:23.272715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cf4a7-1e98-4f72-8bb1-167bd83eb317 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:23.294307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cf4a7-1e98-4f72-8bb1-167bd83eb317 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:23.305902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb16565-b828-4d6b-b057-f5df3425118d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:23.307108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb16565-b828-4d6b-b057-f5df3425118d service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:23.307826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb16565-b828-4d6b-b057-f5df3425118d service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:23.308796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb16565-b828-4d6b-b057-f5df3425118d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:23.325468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb16565-b828-4d6b-b057-f5df3425118d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:23.327812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb16565-b828-4d6b-b057-f5df3425118d service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:23.356968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 845/1695] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:23.432509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675dd49-e778-474f-bfc3-cc810d6ffebb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:23.434381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675dd49-e778-474f-bfc3-cc810d6ffebb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:23.434769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675dd49-e778-474f-bfc3-cc810d6ffebb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:23.444703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 851/1696] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 18 10:47:23 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:23.462978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e349808-384a-4f16-aaae-79f0576e2279 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:23.463368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e349808-384a-4f16-aaae-79f0576e2279 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:23.463944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e349808-384a-4f16-aaae-79f0576e2279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:23.499875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e349808-384a-4f16-aaae-79f0576e2279 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:23.504092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e349808-384a-4f16-aaae-79f0576e2279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:23.521998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 846/1697] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:47:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:23.535568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675dd49-e778-474f-bfc3-cc810d6ffebb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:23.537526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5675dd49-e778-474f-bfc3-cc810d6ffebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:23.628866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675dd49-e778-474f-bfc3-cc810d6ffebb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:23.638727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab071b-26e0-4e9c-a813-442b776d89ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:23.644388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab071b-26e0-4e9c-a813-442b776d89ec service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:23.644388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab071b-26e0-4e9c-a813-442b776d89ec service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:23.644388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab071b-26e0-4e9c-a813-442b776d89ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:23.665969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab071b-26e0-4e9c-a813-442b776d89ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:23.668468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab071b-26e0-4e9c-a813-442b776d89ec service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:23.715561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 852/1698] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:23.981551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:24.142964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:24.210453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38d6a-dd43-46a2-b909-690604317656 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:24.210818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38d6a-dd43-46a2-b909-690604317656 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:24.211165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38d6a-dd43-46a2-b909-690604317656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:24.306760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bd491-054c-4fea-b104-43353f01de38 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:24.307393 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bd491-054c-4fea-b104-43353f01de38 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:24.307994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bd491-054c-4fea-b104-43353f01de38 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:24.312415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 853/1699] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 18 10:47:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:24.332697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:24.333318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:24.333801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:24.347208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-46a38d6a-dd43-46a2-b909-690604317656 None None] MFA Rules not processed for user `c8d33eb0f05f41ca80c2c3b7646a0597`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:24.356873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46a38d6a-dd43-46a2-b909-690604317656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:24.429206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:24.433137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:24.607116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 854/1700] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 18 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:24.609424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:24.611448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:24.631992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af0c5-c518-46db-b383-e55c9bc38dae None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:24.640934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:24.643218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:24.643961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:24.644483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] PATH_INFO: `/v3/projects/51d3c2992fa84e38880b885d48f220f6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:24.647913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d3c2992fa84e38880b885d48f220f6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:24.649820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:47:24.649820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jul 18 10:47:24.649820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: Jul 18 10:47:24.653172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:24.656198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 847/1701] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:24.662542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:24.669426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:24.669825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:24.709232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] MFA Rules not processed for user `e7a738986f87470681c6a578599a089f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:24.711190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:24.849531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d3c2992fa84e38880b885d48f220f6', 'request_id': 'req-c115940d-8130-48cf-a78f-ad4803adc593'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:24.853677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 848/1702] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:24.857434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:24.860518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:24.879187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd667-10e0-4a70-a53c-8a1fbcc441b7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:24.972348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:24.975807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d3c2992fa84e38880b885d48f220f6', 'request_id': 'req-c115940d-8130-48cf-a78f-ad4803adc593'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:24.981544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:24.981938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:24.982163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] PATH_INFO: `/v3/projects/1891541b61be4afbae5eb25c9f718e31` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:24.989942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1891541b61be4afbae5eb25c9f718e31)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:24.999169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:25.022572 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d3c2992fa84e38880b885d48f220f6', 'request_id': 'req-c115940d-8130-48cf-a78f-ad4803adc593'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.075672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] The token cache is being invalidate because project 51d3c2992fa84e38880b885d48f220f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:25.077051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.079739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.088957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 855/1703] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:24 2022] DELETE /identity/v3/projects/51d3c2992fa84e38880b885d48f220f6 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:25.090882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:25.095620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:25.216510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1891541b61be4afbae5eb25c9f718e31', 'request_id': 'req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.225053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1891541b61be4afbae5eb25c9f718e31', 'request_id': 'req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.270377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115940d-8130-48cf-a78f-ad4803adc593 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:25.385539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1891541b61be4afbae5eb25c9f718e31', 'request_id': 'req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.430050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] The token cache is being invalidate because project 1891541b61be4afbae5eb25c9f718e31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:25.431802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.433995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0a13b92-0ea5-4ebb-817e-b17423cb8a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:25.436822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 849/1704] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:24 2022] DELETE /identity/v3/projects/1891541b61be4afbae5eb25c9f718e31 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:25.443865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9221d0-b4fc-4e2a-b39a-5ade2bb27616 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:25.444764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9221d0-b4fc-4e2a-b39a-5ade2bb27616 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:25.445098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9221d0-b4fc-4e2a-b39a-5ade2bb27616 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:25.448596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 850/1705] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 18 10:47:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:25.452047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9221d0-b4fc-4e2a-b39a-5ade2bb27616 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:25.457343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76683c-037b-4fc2-b81a-71415feabdc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:25.458583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8c9221d0-b4fc-4e2a-b39a-5ade2bb27616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:25.459437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76683c-037b-4fc2-b81a-71415feabdc2 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:25.461526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76683c-037b-4fc2-b81a-71415feabdc2 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:25.463381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76683c-037b-4fc2-b81a-71415feabdc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:25.470601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6d76683c-037b-4fc2-b81a-71415feabdc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:25.530207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76683c-037b-4fc2-b81a-71415feabdc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:25.535920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76683c-037b-4fc2-b81a-71415feabdc2 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:25.598775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 856/1706] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:25.610050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e61b87-6bfa-4200-b8f6-e078c2a1a0d9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:25.614325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e61b87-6bfa-4200-b8f6-e078c2a1a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:25.615220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e61b87-6bfa-4200-b8f6-e078c2a1a0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:25.687903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-63e61b87-6bfa-4200-b8f6-e078c2a1a0d9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:25.692875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63e61b87-6bfa-4200-b8f6-e078c2a1a0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:25.820764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9221d0-b4fc-4e2a-b39a-5ade2bb27616 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:25.839114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:25.840941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:25.841878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:25.842433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] PATH_INFO: `/v3/users/e43a7a71ae234b8aa65db59a1b2694ea` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:25.844574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43a7a71ae234b8aa65db59a1b2694ea)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:25.850273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:25.928452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 857/1707] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:25.930701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16411fe3-81a4-4610-914e-abcb1f9e3d7c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:25.931328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16411fe3-81a4-4610-914e-abcb1f9e3d7c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:25.932118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16411fe3-81a4-4610-914e-abcb1f9e3d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:25.976383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-16411fe3-81a4-4610-914e-abcb1f9e3d7c None None] MFA Rules not processed for user `a48b0fdacb2c43649916304e1237f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:25.977979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16411fe3-81a4-4610-914e-abcb1f9e3d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:26.086429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:26.120977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 858/1708] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:26.123967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760a9b-2d47-42df-b140-7e8c955be30f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:26.124338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760a9b-2d47-42df-b140-7e8c955be30f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:26.124628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760a9b-2d47-42df-b140-7e8c955be30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:26.155175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:26.172461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5e760a9b-2d47-42df-b140-7e8c955be30f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:26.173380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43a7a71ae234b8aa65db59a1b2694ea'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:26.174514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e760a9b-2d47-42df-b140-7e8c955be30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:26.193197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 859/1709] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:26.193478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43a7a71ae234b8aa65db59a1b2694ea'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:26.200333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760a9b-2d47-42df-b140-7e8c955be30f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:26.247784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760a9b-2d47-42df-b140-7e8c955be30f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:26.259960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43a7a71ae234b8aa65db59a1b2694ea'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:26.273170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:26.274412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:26.277088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:26.278553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:26.280316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:47:26.280651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:26.283069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:26.321550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 860/1710] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:47:26.325583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:26.354979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e75c5-f1cb-4341-9596-2dd1cfa86481 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:26.384826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:26.386062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:26.386564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:26.387057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:26.389693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:26.399300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:26.477034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43a7a71ae234b8aa65db59a1b2694ea'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:26.478726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43a7a71ae234b8aa65db59a1b2694ea'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:26.508751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 851/1711] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:25 2022] DELETE /identity/v3/users/e43a7a71ae234b8aa65db59a1b2694ea => generated 0 bytes in 1050 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:26.510705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:26.513719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:26.648418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 861/1712] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:26 2022] POST /identity/v3/projects => generated 371 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:26.653967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:26.659076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:26.704968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddfaa9-f236-4ae3-a3b0-d7af4eaf3ffb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:26.841753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:26.842546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:26.842935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:26.843286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:26.848124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:26.875279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:26.878893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:26.888015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:26.911445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 852/1713] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:26.919086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d915a-1faf-414e-8146-7702905e97af admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:26.925126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:26.926022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:26.926713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:26.927132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:26.928036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:26.930668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:27.130090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 862/1714] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:26 2022] POST /identity/v3/users => generated 286 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:27.136574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:27.202216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92e15f-d82e-4003-bed5-460397c07412 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:27.249782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:27.256066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:27.257502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:27.258269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:27.261915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:27.267870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:27.325772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 853/1715] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:27.332918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:27.361840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d049c-bc5b-4895-9f7b-736538fbe555 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:27.377178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86b83c-6e3e-468a-a56f-cb64f21f98c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:27.378718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86b83c-6e3e-468a-a56f-cb64f21f98c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:27.378718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86b83c-6e3e-468a-a56f-cb64f21f98c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:27.378718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86b83c-6e3e-468a-a56f-cb64f21f98c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:27.379390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86b83c-6e3e-468a-a56f-cb64f21f98c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:27.386900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86b83c-6e3e-468a-a56f-cb64f21f98c0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:27.426782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 863/1716] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:27.429552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:27.464492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267fae8-72a5-4e81-9316-d7f755671097 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:27.483295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abe3aa-5d26-4730-878b-96a65a647ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:27.485291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abe3aa-5d26-4730-878b-96a65a647ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:27.485626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abe3aa-5d26-4730-878b-96a65a647ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:27.485914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abe3aa-5d26-4730-878b-96a65a647ad1 admin admin] PATH_INFO: `/v3/projects/6cd30e8c51cf4665b4c5f5262b56053f/users/c597f753e1b542b794c3c68d9cbba8b7/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:27.519230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abe3aa-5d26-4730-878b-96a65a647ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd30e8c51cf4665b4c5f5262b56053f, user_id=c597f753e1b542b794c3c68d9cbba8b7, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:27.523950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abe3aa-5d26-4730-878b-96a65a647ad1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:27.658498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 854/1717] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:27 2022] PUT /identity/v3/projects/6cd30e8c51cf4665b4c5f5262b56053f/users/c597f753e1b542b794c3c68d9cbba8b7/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:27.664327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e6f0f-f273-44e3-b019-08cb4430f6da None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:27.666066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e6f0f-f273-44e3-b019-08cb4430f6da None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:27.666607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e6f0f-f273-44e3-b019-08cb4430f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:27.673649 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:27.692650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-001e6f0f-f273-44e3-b019-08cb4430f6da None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:27.694243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-001e6f0f-f273-44e3-b019-08cb4430f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:27.909190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267128c-52e0-4065-9256-f232bac78296 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:27.909190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267128c-52e0-4065-9256-f232bac78296 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:27.909190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267128c-52e0-4065-9256-f232bac78296 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:27.915482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 855/1718] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:47:27 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:27.924604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:27.925456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:27.925840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:27.931929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 864/1719] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:27.983459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:27.989863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:27.997711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e6f0f-f273-44e3-b019-08cb4430f6da None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:28.196289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e6f0f-f273-44e3-b019-08cb4430f6da None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:28.203989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 856/1720] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:47:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:28.210424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:28.213111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:28.215540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:28.215691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:28.217555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:28.265248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:28.266529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:28.292287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d21ea7-d208-4eb3-95a7-b99b9aafb46f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:28.297244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:28.309586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55307a82-dc33-4275-8b10-5b7831ec4cfb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:28.309915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:28.310916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55307a82-dc33-4275-8b10-5b7831ec4cfb service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:28.311485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55307a82-dc33-4275-8b10-5b7831ec4cfb service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:28.312991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55307a82-dc33-4275-8b10-5b7831ec4cfb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:28.351466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55307a82-dc33-4275-8b10-5b7831ec4cfb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:28.359953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55307a82-dc33-4275-8b10-5b7831ec4cfb service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:28.380042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 865/1721] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:28.396850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 857/1722] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:29.741644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:29.889893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:29.893117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:29.929391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309aaab-7962-40d3-a9da-d82d589019eb service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:29.942919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:29.946999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:29.949632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:29.950248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] PATH_INFO: `/v3/projects/1ccce20c1e66479191cb1f4ca2bc9f54` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:29.952542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccce20c1e66479191cb1f4ca2bc9f54)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:29.954962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:29.994048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756214e-bd75-4128-89f0-8ceed43448d0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:29.994935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756214e-bd75-4128-89f0-8ceed43448d0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:29.995219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756214e-bd75-4128-89f0-8ceed43448d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:30.050267 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8756214e-bd75-4128-89f0-8ceed43448d0 None None] MFA Rules not processed for user `3dbb489073d94425ad1a1b350768c428`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:30.057437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8756214e-bd75-4128-89f0-8ceed43448d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:30.325996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 858/1723] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:30.336439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccce20c1e66479191cb1f4ca2bc9f54', 'request_id': 'req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:30.501920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccce20c1e66479191cb1f4ca2bc9f54', 'request_id': 'req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:30.526837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccce20c1e66479191cb1f4ca2bc9f54', 'request_id': 'req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:30.562062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] The token cache is being invalidate because project 1ccce20c1e66479191cb1f4ca2bc9f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:30.562062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:30.562062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d6ad4dd-65c7-4d43-bf8c-efa22356f0c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:30.571505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 866/1724] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:29 2022] DELETE /identity/v3/projects/1ccce20c1e66479191cb1f4ca2bc9f54 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:30.574591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabf3dd-38eb-4ff9-ba4e-bdb4f3059805 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:30.576131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabf3dd-38eb-4ff9-ba4e-bdb4f3059805 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:30.577270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabf3dd-38eb-4ff9-ba4e-bdb4f3059805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:30.675706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fcabf3dd-38eb-4ff9-ba4e-bdb4f3059805 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:30.678182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fcabf3dd-38eb-4ff9-ba4e-bdb4f3059805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:30.989083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 859/1725] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:30.999002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f712412-ef2a-4470-92f9-9c71588b2d2c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:31.000515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f712412-ef2a-4470-92f9-9c71588b2d2c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:31.001519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f712412-ef2a-4470-92f9-9c71588b2d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:31.047818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f712412-ef2a-4470-92f9-9c71588b2d2c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:31.054848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f712412-ef2a-4470-92f9-9c71588b2d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:31.083736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 867/1726] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:31.090800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabf3dd-38eb-4ff9-ba4e-bdb4f3059805 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:31.130887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabf3dd-38eb-4ff9-ba4e-bdb4f3059805 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:31.145032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:31.151905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:31.151905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:31.153067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:31.158236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:47:31.160451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:31.163881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:31.204480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 860/1727] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:47:31.227187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f712412-ef2a-4470-92f9-9c71588b2d2c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:31.229456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:31.321037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f712412-ef2a-4470-92f9-9c71588b2d2c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:31.352140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:31.353786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:31.357078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:31.358007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:31.368370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:31.378930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:31.636281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 868/1728] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:31 2022] POST /identity/v3/projects => generated 387 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:31.640029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:31.662616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d67591-288f-497a-9d1a-31e074c5a961 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:31.899086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:32.001722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:32.002594 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:32.002941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:32.003244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:32.019104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:32.032684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:32.183608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 861/1729] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:31 2022] POST /identity/v3/users => generated 294 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:47:32.190264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:32.235465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425846-0879-40e3-928e-5e41f319dc52 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:32.252873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:32.254575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:32.255279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:32.256301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:32.261013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:32.276409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:32.339498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 869/1730] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:32.345162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:32.366126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1862725-1a1a-438c-9d59-12ddb1a4a538 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:32.393419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6418bf-a8fe-4d7b-9657-04506919efc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:32.396212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6418bf-a8fe-4d7b-9657-04506919efc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:32.398937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6418bf-a8fe-4d7b-9657-04506919efc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:32.399990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6418bf-a8fe-4d7b-9657-04506919efc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:32.406159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6418bf-a8fe-4d7b-9657-04506919efc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:32.413546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6418bf-a8fe-4d7b-9657-04506919efc4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:32.454196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 862/1731] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:32.454963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:32.487153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253537-d39e-48b4-92ac-8a7f88bbbfef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:32.505980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:32.509664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:32.510853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:32.511627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] PATH_INFO: `/v3/projects/345b7d8c3b9e4286b6d6d35f9cff572a/users/a629114d8a4b45bf980654db3f9235b2/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:32.566772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] RBAC: Authorizing `identity:create_grant(project_id=345b7d8c3b9e4286b6d6d35f9cff572a, user_id=a629114d8a4b45bf980654db3f9235b2, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:32.578381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:32.872633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468ba8c-d60d-4247-ae06-5d05fbdfd48f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:32.873397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468ba8c-d60d-4247-ae06-5d05fbdfd48f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:32.874065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468ba8c-d60d-4247-ae06-5d05fbdfd48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:32.877444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 870/1732] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:32 2022] PUT /identity/v3/projects/345b7d8c3b9e4286b6d6d35f9cff572a/users/a629114d8a4b45bf980654db3f9235b2/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:47:33.018368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b468ba8c-d60d-4247-ae06-5d05fbdfd48f None None] MFA Rules not processed for user `a629114d8a4b45bf980654db3f9235b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:33.022609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b468ba8c-d60d-4247-ae06-5d05fbdfd48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:33.246948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:33.253679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:33.294008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 863/1733] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:33.296659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468ba8c-d60d-4247-ae06-5d05fbdfd48f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:33.557518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468ba8c-d60d-4247-ae06-5d05fbdfd48f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:33.557518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977daf8b-2c11-43b2-8e31-e269878cec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:33.557518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977daf8b-2c11-43b2-8e31-e269878cec25 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:33.557518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977daf8b-2c11-43b2-8e31-e269878cec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:33.560330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977daf8b-2c11-43b2-8e31-e269878cec25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:33.560330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977daf8b-2c11-43b2-8e31-e269878cec25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:33.560330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977daf8b-2c11-43b2-8e31-e269878cec25 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:33.579603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c822-5ecd-4421-92f6-575088a0ddef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:33.607150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:33.634337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:33.634337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:33.634337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] PATH_INFO: `/v3/users/0b1882bcbe8543f18d617158f582aeb6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:33.634337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b1882bcbe8543f18d617158f582aeb6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:33.636902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:33.752371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 864/1734] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:33 2022] POST /identity/v3/projects => generated 385 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:33.763478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5ecd-a1db-45d5-ab33-ddc07066b34c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:33.767520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5ecd-a1db-45d5-ab33-ddc07066b34c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:33.767869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5ecd-a1db-45d5-ab33-ddc07066b34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:33.921885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c78a5ecd-a1db-45d5-ab33-ddc07066b34c None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:33.923983 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c78a5ecd-a1db-45d5-ab33-ddc07066b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:34.165621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b1882bcbe8543f18d617158f582aeb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:34.233085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b1882bcbe8543f18d617158f582aeb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:34.254907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 865/1735] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:47:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:34.255991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a5ecd-a1db-45d5-ab33-ddc07066b34c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:34.399477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b1882bcbe8543f18d617158f582aeb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:34.401245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b1882bcbe8543f18d617158f582aeb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:34.402341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d61385a0-174a-47cb-bb5b-0c7a86c559b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b1882bcbe8543f18d617158f582aeb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:34.452536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:34.509603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 871/1736] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:33 2022] DELETE /identity/v3/users/0b1882bcbe8543f18d617158f582aeb6 => generated 0 bytes in 1264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:34.513249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30328c-0fdd-4c30-b3ed-fc27b8d9b7a8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:34.519619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30328c-0fdd-4c30-b3ed-fc27b8d9b7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:34.519976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30328c-0fdd-4c30-b3ed-fc27b8d9b7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:34.606970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a5ecd-a1db-45d5-ab33-ddc07066b34c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:34.608550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2f30328c-0fdd-4c30-b3ed-fc27b8d9b7a8 None None] MFA Rules not processed for user `c597f753e1b542b794c3c68d9cbba8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:34.610150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2f30328c-0fdd-4c30-b3ed-fc27b8d9b7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:34.849068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:34.852415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:34.853385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:34.854573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:34.857646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:34.864229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:34.940629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 872/1737] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:34.942609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:34.947284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:34.947968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:35.020488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] MFA Rules not processed for user `7e032c8e5b024fa7a41cd1090e0f1ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:35.021988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:35.052253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 866/1738] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:34 2022] POST /identity/v3/users => generated 293 bytes in 789 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:35.055693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:35.068707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:35.108355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edeb95-01d7-4e19-9098-f2fa15cb412d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:35.123323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:35.127148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:35.127539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:35.129136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:35.133943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:35.139165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:35.147785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=14,keystone:SELECT=82,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:35.219635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 873/1739] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:35.219635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:35.226598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:35.390390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 867/1740] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:35 2022] POST /identity/v3/projects => generated 373 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:35.391679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:35.405215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:35.577213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b9ab4-d4db-4907-8e71-fe21bffc1daa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:35.714119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13dd64-616b-44e5-90cc-b826e66b5eca admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:35.725297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:35.727203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:35.728079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:35.729144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:35.730644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:35.733865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:35.755304 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 868/1741] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:35.758601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:35.778321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdfb42-0e40-477b-b0a2-a161470daa4b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:35.808266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:35.810534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:35.810959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:35.811353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:35.812239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:35.815199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:35.851104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:35.852691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:35.853419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:35.855265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:35.861051 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:36.029453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 869/1742] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:35 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:36.046001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:36.111131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788046f2-7e24-441d-aa7f-3283d1baf341 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:36.131038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:36.134177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:36.136618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:36.137536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:36.144212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:36.147454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:36.154961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:36.176684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:36.216538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 874/1743] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:47:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 999 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:47:36.219739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:36.243116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 870/1744] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:36.246542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:36.252220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992af770-e023-47ee-9b51-ad78c085c7d5 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:36.262328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:36.263607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:36.264539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:36.265612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:36.267108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:36.272076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:36.334159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 875/1745] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:36.341229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded99131-c06c-41da-ae0b-ffd5e4ac293d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:36.343884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:36.386756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec2bf5-14c1-46b4-a6a9-b6dbc81448a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:36.392795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec2bf5-14c1-46b4-a6a9-b6dbc81448a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:36.399179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec2bf5-14c1-46b4-a6a9-b6dbc81448a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:36.399179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec2bf5-14c1-46b4-a6a9-b6dbc81448a1 admin admin] PATH_INFO: `/v3/projects/ef0c65bbe263445d93ba34b837cde3be/users/e056cce76b4947e5b606a0741b6121a7/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:36.421545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a58fdf-c2f2-4c09-bde8-ce100921e7de admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:36.431743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:36.439290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:36.458937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:36.459764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] PATH_INFO: `/v3/projects/811a92f9ef894cfc8e2de198435000e4/users/ea9272ca9692490cae3d889676b61ba9/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:36.509367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] RBAC: Authorizing `identity:create_grant(project_id=811a92f9ef894cfc8e2de198435000e4, user_id=ea9272ca9692490cae3d889676b61ba9, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:36.521100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:36.526308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:36.561263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec2bf5-14c1-46b4-a6a9-b6dbc81448a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0c65bbe263445d93ba34b837cde3be, user_id=e056cce76b4947e5b606a0741b6121a7, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:36.575091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec2bf5-14c1-46b4-a6a9-b6dbc81448a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:36.685641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 876/1746] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:36 2022] PUT /identity/v3/projects/811a92f9ef894cfc8e2de198435000e4/users/ea9272ca9692490cae3d889676b61ba9/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:36.693536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:36.764191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 871/1747] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:36 2022] PUT /identity/v3/projects/ef0c65bbe263445d93ba34b837cde3be/users/e056cce76b4947e5b606a0741b6121a7/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:36.780816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ced0-2213-4363-ae85-f23fe254842a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:36.781286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ced0-2213-4363-ae85-f23fe254842a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:36.781788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ced0-2213-4363-ae85-f23fe254842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:36.989886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-06f7ced0-2213-4363-ae85-f23fe254842a None None] MFA Rules not processed for user `e056cce76b4947e5b606a0741b6121a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:36.992100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06f7ced0-2213-4363-ae85-f23fe254842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:37.091886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad009d6-7d95-4f66-9ca5-508d3df4b44c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:37.304023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 872/1748] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:37.315573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7ced0-2213-4363-ae85-f23fe254842a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:37.472627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:37.524587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:37.527435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:37.528265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:37.528874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:37.538577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:37.542175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:37.634182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 877/1749] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 942 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:37.639957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:37.693045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:37.725473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9b25d-6ab5-4dfa-a273-90b7852d11ea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:37.738011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7ced0-2213-4363-ae85-f23fe254842a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:37.760476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:37.771146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:37.771939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:37.772767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:37.773392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:37.775535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:37.785944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:37.786913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:37.790724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] PATH_INFO: `/v3/projects/811a92f9ef894cfc8e2de198435000e4/users/ea9272ca9692490cae3d889676b61ba9/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:37.814015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:37.831385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:37.833104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811a92f9ef894cfc8e2de198435000e4, user_id=ea9272ca9692490cae3d889676b61ba9, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:37.866768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:37.945931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 878/1750] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:37 2022] PUT /identity/v3/projects/811a92f9ef894cfc8e2de198435000e4/users/ea9272ca9692490cae3d889676b61ba9/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:38.106629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:38.109476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 873/1751] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:37 2022] POST /identity/v3/projects => generated 387 bytes in 795 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:38.306392 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ea7fe-54dc-4ff4-a630-313b32244d93 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:38.514012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb614dfa-7ed5-4c28-b924-492f0270389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:38.517867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb614dfa-7ed5-4c28-b924-492f0270389e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:38.521598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb614dfa-7ed5-4c28-b924-492f0270389e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:38.522306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb614dfa-7ed5-4c28-b924-492f0270389e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:38.523765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb614dfa-7ed5-4c28-b924-492f0270389e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:38.554808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb614dfa-7ed5-4c28-b924-492f0270389e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:38.792872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 879/1752] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:38 2022] POST /identity/v3/users => generated 300 bytes in 688 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:47:38.798936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:38.859997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6665f9-535b-40c9-b510-7e7e2e7a66f9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:38.883643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:38.884108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:38.884503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:38.925954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:38.929767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:38.931068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] MFA Rules not processed for user `3954fd4abf6b4c3a82280c762292a3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:38.933269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:38.935541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:38.937740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:38.940062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:38.947839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:38.993937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 874/1753] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:38.998704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:39.095230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5251841-4895-45e5-83ed-cca307af0740 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:39.125639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:39.127065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:39.127524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:39.128029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] PATH_INFO: `/v3/projects/63a9280dd3e14777a2ad38891a1096d2/users/9173677075c84c97bfe993bacd35837d/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:39.216944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a9280dd3e14777a2ad38891a1096d2, user_id=9173677075c84c97bfe993bacd35837d, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:39.225104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:39.232269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:39.236860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 880/1754] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:39.237807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:39.247997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:39.299405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:39.310958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7ec7-1c74-4c4a-bd91-b8724ce128c1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:39.331424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:39.337794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:39.338222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:39.341137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] PATH_INFO: `/v3/projects/1933792aea1c42a9b766cfd51820c7e1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:39.343222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1933792aea1c42a9b766cfd51820c7e1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:39.345388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:39.410130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 875/1755] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:38 2022] PUT /identity/v3/projects/63a9280dd3e14777a2ad38891a1096d2/users/9173677075c84c97bfe993bacd35837d/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:39.415170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:39.639446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f107-4948-470e-bd71-fbea760b6844 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:39.654938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79278d-70d6-424d-a0b9-cc1581ae1e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:39.655658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79278d-70d6-424d-a0b9-cc1581ae1e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:39.656134 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79278d-70d6-424d-a0b9-cc1581ae1e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:39.656482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79278d-70d6-424d-a0b9-cc1581ae1e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:39.657299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79278d-70d6-424d-a0b9-cc1581ae1e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:39.661798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79278d-70d6-424d-a0b9-cc1581ae1e97 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:39.686227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 876/1756] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:39.694288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5a92-57ed-4beb-8e02-33693584602a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:39.697759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5a92-57ed-4beb-8e02-33693584602a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:39.697759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5a92-57ed-4beb-8e02-33693584602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:39.748384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-394c5a92-57ed-4beb-8e02-33693584602a None None] MFA Rules not processed for user `a48b0fdacb2c43649916304e1237f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:39.760022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-394c5a92-57ed-4beb-8e02-33693584602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:39.827840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1933792aea1c42a9b766cfd51820c7e1', 'request_id': 'req-bde89ef2-1729-4398-afa4-e97b4f62d514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:40.038536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1933792aea1c42a9b766cfd51820c7e1', 'request_id': 'req-bde89ef2-1729-4398-afa4-e97b4f62d514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:40.039932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 877/1757] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Jul 18 10:47:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:40.042512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c5a92-57ed-4beb-8e02-33693584602a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:40.085655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1933792aea1c42a9b766cfd51820c7e1', 'request_id': 'req-bde89ef2-1729-4398-afa4-e97b4f62d514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:40.199692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] The token cache is being invalidate because project 1933792aea1c42a9b766cfd51820c7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:40.200041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:40.201243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:40.208180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 881/1758] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:39 2022] DELETE /identity/v3/projects/1933792aea1c42a9b766cfd51820c7e1 => generated 0 bytes in 968 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:40.208180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:40.214521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:40.274374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89ef2-1729-4398-afa4-e97b4f62d514 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:40.302950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:40.304227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:40.304664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:40.305192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:40.324871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:40.370498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:40.382132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:40.434757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c5a92-57ed-4beb-8e02-33693584602a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:40.473129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eed497-8cf5-4f1f-856e-3bdc05882fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:40.482595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 882/1759] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:40.489703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:40.489919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eed497-8cf5-4f1f-856e-3bdc05882fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:40.491963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eed497-8cf5-4f1f-856e-3bdc05882fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:40.494226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:40.496371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eed497-8cf5-4f1f-856e-3bdc05882fe3 admin admin] PATH_INFO: `/v3/projects/63a9280dd3e14777a2ad38891a1096d2/users/9173677075c84c97bfe993bacd35837d/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:40.522363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eed497-8cf5-4f1f-856e-3bdc05882fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a9280dd3e14777a2ad38891a1096d2, user_id=9173677075c84c97bfe993bacd35837d, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:40.563091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eed497-8cf5-4f1f-856e-3bdc05882fe3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:40.588538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e998b-bb42-4dbb-a05f-19cd78b4e2d9 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:40.611710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:40.612840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:40.613252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:40.614130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] PATH_INFO: `/v3/users/c8d33eb0f05f41ca80c2c3b7646a0597` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:40.619160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d33eb0f05f41ca80c2c3b7646a0597)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:40.626241 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:40.642950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 878/1760] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:40 2022] PUT /identity/v3/projects/63a9280dd3e14777a2ad38891a1096d2/users/9173677075c84c97bfe993bacd35837d/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:40.670350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:40.672085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:40.672804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:40.792906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] MFA Rules not processed for user `9173677075c84c97bfe993bacd35837d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:40.795416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:41.004700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 879/1761] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:41.044208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:41.046597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:41.051503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d33eb0f05f41ca80c2c3b7646a0597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:41.146848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e61ee8-b25f-4993-b720-bf4e6df8d72b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:41.153317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d33eb0f05f41ca80c2c3b7646a0597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:41.184724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:41.186684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:41.187936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:41.191029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:41.210715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:41.215876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:41.236347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:41.292328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d33eb0f05f41ca80c2c3b7646a0597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:41.294334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d33eb0f05f41ca80c2c3b7646a0597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:41.297038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d33eb0f05f41ca80c2c3b7646a0597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:41.308523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 880/1762] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:41.382785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 883/1763] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:40 2022] DELETE /identity/v3/users/c8d33eb0f05f41ca80c2c3b7646a0597 => generated 0 bytes in 892 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:44.068117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:44.182217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:44.302381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:44.304912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:44.391309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27d47f-b99f-4b6b-ac95-2026c18b5608 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:44.546256 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:44.701750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ec89f-9094-4e15-8175-d5c301352be0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:44.718710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63473078-ba02-4ee8-baa0-bfafd847c812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:44.719399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63473078-ba02-4ee8-baa0-bfafd847c812 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:44.734031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63473078-ba02-4ee8-baa0-bfafd847c812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:44.734031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63473078-ba02-4ee8-baa0-bfafd847c812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:44.734313 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63473078-ba02-4ee8-baa0-bfafd847c812 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:44.751299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:44.752314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:44.752745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:44.753054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] PATH_INFO: `/v3/projects/295a4c37c02d4c0ab7fdab38198bb254` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:44.762209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] RBAC: Authorizing `identity:delete_project(project_id=295a4c37c02d4c0ab7fdab38198bb254)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:44.764481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:44.952305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63473078-ba02-4ee8-baa0-bfafd847c812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:44.959338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63473078-ba02-4ee8-baa0-bfafd847c812 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:44.996132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 884/1764] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:44.999177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:45.000298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:45.007100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:45.110834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] MFA Rules not processed for user `ea9272ca9692490cae3d889676b61ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:45.112671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:45.118418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '295a4c37c02d4c0ab7fdab38198bb254', 'request_id': 'req-1ffd40f3-9726-460b-a880-598ce35bc85f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:45.134068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '295a4c37c02d4c0ab7fdab38198bb254', 'request_id': 'req-1ffd40f3-9726-460b-a880-598ce35bc85f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:45.228503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '295a4c37c02d4c0ab7fdab38198bb254', 'request_id': 'req-1ffd40f3-9726-460b-a880-598ce35bc85f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:45.250359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 885/1765] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:45.268250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:45.270193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:45.282710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] The token cache is being invalidate because project 295a4c37c02d4c0ab7fdab38198bb254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:45.285828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:45.294151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ffd40f3-9726-460b-a880-598ce35bc85f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:45.306385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 881/1766] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:44 2022] DELETE /identity/v3/projects/295a4c37c02d4c0ab7fdab38198bb254 => generated 0 bytes in 1231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:45.310504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf0f05-e350-496e-aec2-a9a1c4518e82 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:45.326636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf0f05-e350-496e-aec2-a9a1c4518e82 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:45.326636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf0f05-e350-496e-aec2-a9a1c4518e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:45.357812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c774b-f2d0-489c-9ac6-257396325a54 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:45.363818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64082af3-d58a-4055-9bf4-333124e5f06b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:45.364983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64082af3-d58a-4055-9bf4-333124e5f06b service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:45.365320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64082af3-d58a-4055-9bf4-333124e5f06b service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:45.366112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64082af3-d58a-4055-9bf4-333124e5f06b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:45.368973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64082af3-d58a-4055-9bf4-333124e5f06b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:45.436600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1faf0f05-e350-496e-aec2-a9a1c4518e82 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:45.441061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1faf0f05-e350-496e-aec2-a9a1c4518e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:45.519328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64082af3-d58a-4055-9bf4-333124e5f06b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:45.522852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64082af3-d58a-4055-9bf4-333124e5f06b service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:45.560239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 886/1767] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:45.717786 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 882/1768] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:45.739157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fe5c7-5942-4ca7-b38b-3f2dfc918a71 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:45.739157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fe5c7-5942-4ca7-b38b-3f2dfc918a71 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:45.739157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fe5c7-5942-4ca7-b38b-3f2dfc918a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:45.817750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d8fe5c7-5942-4ca7-b38b-3f2dfc918a71 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:45.830016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d8fe5c7-5942-4ca7-b38b-3f2dfc918a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:45.879184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 887/1769] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:45.880904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf0f05-e350-496e-aec2-a9a1c4518e82 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:45.912380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf0f05-e350-496e-aec2-a9a1c4518e82 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:45.937770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:45.939184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:45.940772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:45.941305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:45.944015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:47:45.944465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:45.951800 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:46.023729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 883/1770] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:47:46.028361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fe5c7-5942-4ca7-b38b-3f2dfc918a71 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:46.062246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fe5c7-5942-4ca7-b38b-3f2dfc918a71 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:46.090019 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:46.092451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:46.095259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:46.095970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:46.097591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:46.105083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:46.405300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:46.406498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 888/1771] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:46 2022] POST /identity/v3/projects => generated 357 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:46.471135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee1fdf-d026-4e63-8e52-3d5da0619bc3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:46.831767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:46.839540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:46.845527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:46.845949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:46.863344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:46.863344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:47.024867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 884/1772] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:46 2022] POST /identity/v3/users => generated 279 bytes in 621 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:47.030944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:47.122624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0ffa-6179-448f-88d2-bce5f8bc75cf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:47.223787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:47.230008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:47.234093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:47.251582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:47.261453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:47.267739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:47.354646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 889/1773] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:47.360567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:47.382444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccf5d0-d2cc-4cb5-8203-63d42c1d0cf4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:47.401631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1c7d0-f15c-45f2-b7a2-729e624d7495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:47.403653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1c7d0-f15c-45f2-b7a2-729e624d7495 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:47.404529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1c7d0-f15c-45f2-b7a2-729e624d7495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:47.405156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1c7d0-f15c-45f2-b7a2-729e624d7495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:47.407198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1c7d0-f15c-45f2-b7a2-729e624d7495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:47.412131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1c7d0-f15c-45f2-b7a2-729e624d7495 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:47.447936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:47.448634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 885/1774] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:47.479124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed8da3-33af-4106-bd4e-16a6e9edaa33 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:47.486654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:47.489108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:47.499662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:47.502636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] PATH_INFO: `/v3/projects/d683c9972fdd4996bf6b82aa16f1b50a/users/a52fecde74244b009bf48d41c8049f84/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:47.512529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:47.537593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d683c9972fdd4996bf6b82aa16f1b50a, user_id=a52fecde74244b009bf48d41c8049f84, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:47.537593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:47.852460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 890/1775] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:47 2022] PUT /identity/v3/projects/d683c9972fdd4996bf6b82aa16f1b50a/users/a52fecde74244b009bf48d41c8049f84/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:47:47.877078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c9499-b91e-4edf-b625-b82385dbf351 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:47.878965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c9499-b91e-4edf-b625-b82385dbf351 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:47.881615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c9499-b91e-4edf-b625-b82385dbf351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:47.996477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-460c9499-b91e-4edf-b625-b82385dbf351 None None] MFA Rules not processed for user `a52fecde74244b009bf48d41c8049f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:47.998154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-460c9499-b91e-4edf-b625-b82385dbf351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:48.265168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 886/1776] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:48.272213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:48.307957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21b1ff-caa8-44a0-bfd2-45064f7b936d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:48.308494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21b1ff-caa8-44a0-bfd2-45064f7b936d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:48.309052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21b1ff-caa8-44a0-bfd2-45064f7b936d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:48.316587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 887/1777] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:47:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:48.329974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0e36-74ae-4d3c-8a15-e1dc60e1e25b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:48.331582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0e36-74ae-4d3c-8a15-e1dc60e1e25b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:48.331894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0e36-74ae-4d3c-8a15-e1dc60e1e25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:48.456630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2c7a0e36-74ae-4d3c-8a15-e1dc60e1e25b None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:48.470677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c7a0e36-74ae-4d3c-8a15-e1dc60e1e25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:48.682522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393873a-dc5a-40b7-9010-81e1ec3c137f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:48.704005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:48.718025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:48.719534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:48.720494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:48.724131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:48.733433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:48.791738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 888/1778] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:48.949873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a0e36-74ae-4d3c-8a15-e1dc60e1e25b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:48.969203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 891/1779] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:48 2022] POST /identity/v3/projects => generated 357 bytes in 697 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:47:49.008677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a0e36-74ae-4d3c-8a15-e1dc60e1e25b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:49.305387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:49.306184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:49.306470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:49.307891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:49.308806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:49.311993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:49.456828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:49.472635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 889/1780] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:48 2022] POST /identity/v3/users => generated 285 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:49.479065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790208d-bbef-40b5-9c88-1cf05bb753e1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:49.486462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:49.490070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:49.490070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:49.490070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:49.490070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:49.491782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:49.516823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:49.526595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:49.527604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 892/1781] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:49.545718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2924f-b885-4fc1-8694-65b19eaf1265 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:49.545718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:49.545718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:49.545718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:49.546230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] PATH_INFO: `/v3/projects/054875c2a7cc4838bd05084826ba0683/users/0c85ae8409f24d3883c5c0a48a492956/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:49.563709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=054875c2a7cc4838bd05084826ba0683, user_id=0c85ae8409f24d3883c5c0a48a492956, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:49.580083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:49.744887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 890/1782] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:49 2022] PUT /identity/v3/projects/054875c2a7cc4838bd05084826ba0683/users/0c85ae8409f24d3883c5c0a48a492956/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:49.778667 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:49.789881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:49.792667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:49.802102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3a18-9073-4767-979c-1c33afcad698 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:49.929533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d967b-4877-450b-8c30-ba345b1c1317 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:49.953027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:49.953027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:49.953027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:49.953027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:49.963075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:50.233178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:50.244341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:50.245654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:50.246158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:50.246559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:50.266290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:50.280904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:50.317491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:50.323482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:50.332907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 893/1783] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 574 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:50.355050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:50.371031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 891/1784] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:50.380059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:50.389340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:50.397544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3e2c6-13ed-4476-a5a7-4629cd38da62 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:50.408310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b555289-aed1-475e-89e2-63d01d78d1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:50.410651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b555289-aed1-475e-89e2-63d01d78d1c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:50.411669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b555289-aed1-475e-89e2-63d01d78d1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:50.423266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b555289-aed1-475e-89e2-63d01d78d1c7 admin admin] PATH_INFO: `/v3/projects/054875c2a7cc4838bd05084826ba0683/users/0c85ae8409f24d3883c5c0a48a492956/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:50.429181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b555289-aed1-475e-89e2-63d01d78d1c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=054875c2a7cc4838bd05084826ba0683, user_id=0c85ae8409f24d3883c5c0a48a492956, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:50.437451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b555289-aed1-475e-89e2-63d01d78d1c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:50.511670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 894/1785] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:50 2022] PUT /identity/v3/projects/054875c2a7cc4838bd05084826ba0683/users/0c85ae8409f24d3883c5c0a48a492956/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:47:50.518677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:50.522836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:50.525006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:50.626422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:50.647032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] MFA Rules not processed for user `0c85ae8409f24d3883c5c0a48a492956`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:50.654072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:50.794792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e4b88-1292-400a-b3cd-6ef7ca644803 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:50.976990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:50.978550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:50.979325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:50.979960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:51.001990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:51.006479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:51.009511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 895/1786] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:51.028126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:51.031609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 892/1787] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:51.035237 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:51.225938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecaa66-b7c8-482e-aca5-02b42451b9f3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:51.225938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e23d-2fbf-4e93-aee7-f6ee048ddae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:51.226378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e23d-2fbf-4e93-aee7-f6ee048ddae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:51.227010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e23d-2fbf-4e93-aee7-f6ee048ddae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:51.227312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e23d-2fbf-4e93-aee7-f6ee048ddae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:51.255569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e23d-2fbf-4e93-aee7-f6ee048ddae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:51.259182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e23d-2fbf-4e93-aee7-f6ee048ddae7 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:51.288983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 896/1788] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:51 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:51.807950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:51.850003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e422e67-550a-4ebb-bca6-50dafc96e02f service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:51.864657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:51.866347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:51.866934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:51.867586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:51.972680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e4c06-0c80-4456-b5c4-7bcb87ab4038 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:51.973361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e4c06-0c80-4456-b5c4-7bcb87ab4038 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:51.974017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e4c06-0c80-4456-b5c4-7bcb87ab4038 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:51.977040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 897/1789] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:47:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:47:52.181720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ca49-78c1-4b5f-b305-a78aee75a833 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:52.182768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ca49-78c1-4b5f-b305-a78aee75a833 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:52.183252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ca49-78c1-4b5f-b305-a78aee75a833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:52.212118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:52.217789 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:52.239860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-16c0ca49-78c1-4b5f-b305-a78aee75a833 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:52.254628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16c0ca49-78c1-4b5f-b305-a78aee75a833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:52.265681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 893/1790] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:51 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:52.306502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 898/1791] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:47:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:47:53.152958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:53.367222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:53.369106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:53.421617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db3c3a-b03e-44b5-bbbd-8601a5f57e00 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:53.463139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:53.468655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:53.476969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:53.477839 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:53.517193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:53.524885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:53.551103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 894/1792] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:54.996343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ca49-78c1-4b5f-b305-a78aee75a833 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:55.008716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ca49-78c1-4b5f-b305-a78aee75a833 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:55.013347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1408c-fe10-4572-bd1c-809c7934739d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:55.014163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1408c-fe10-4572-bd1c-809c7934739d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:55.014639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1408c-fe10-4572-bd1c-809c7934739d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:55.014925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1408c-fe10-4572-bd1c-809c7934739d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:55.017093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ee1408c-fe10-4572-bd1c-809c7934739d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:55.076516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:55.110940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1408c-fe10-4572-bd1c-809c7934739d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:55.117127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1408c-fe10-4572-bd1c-809c7934739d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:55.146245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 899/1793] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:02.792940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:02.794933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:02.817778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:02.895419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dccc64-f303-49fd-b635-1d9b4cd1a6cc service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:02.903455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:02.905037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:02.905553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:02.906066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] PATH_INFO: `/v3/users/a629114d8a4b45bf980654db3f9235b2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:02.908949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a629114d8a4b45bf980654db3f9235b2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:02.912824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:03.309758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a629114d8a4b45bf980654db3f9235b2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:03.320854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a629114d8a4b45bf980654db3f9235b2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:03.343360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a629114d8a4b45bf980654db3f9235b2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:03.526924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a629114d8a4b45bf980654db3f9235b2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:03.527566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a629114d8a4b45bf980654db3f9235b2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:03.536165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 895/1794] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:02 2022] DELETE /identity/v3/users/a629114d8a4b45bf980654db3f9235b2 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:03.543278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50355131-53f2-4a69-845a-3d9a10442e29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:03.544096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50355131-53f2-4a69-845a-3d9a10442e29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:03.544096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50355131-53f2-4a69-845a-3d9a10442e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:03.562823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-50355131-53f2-4a69-845a-3d9a10442e29 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:03.564062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50355131-53f2-4a69-845a-3d9a10442e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:03.644044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 900/1795] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:03.655733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:03.659874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:03.745651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92cba5-d60c-458a-a124-1a81eea8d921 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:03.749697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef81968-1089-47e0-ac7e-4cf24037e7eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:03.750851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef81968-1089-47e0-ac7e-4cf24037e7eb service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:03.751054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef81968-1089-47e0-ac7e-4cf24037e7eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:03.751340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef81968-1089-47e0-ac7e-4cf24037e7eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:03.763021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef81968-1089-47e0-ac7e-4cf24037e7eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:03.764500 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef81968-1089-47e0-ac7e-4cf24037e7eb service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:03.776976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92643840-c8f5-40dd-88c3-033cfaa43737 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:03.777669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92643840-c8f5-40dd-88c3-033cfaa43737 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:03.778263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92643840-c8f5-40dd-88c3-033cfaa43737 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:03.779122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 896/1796] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:03.780837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 901/1797] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:48:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:48:03.793075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:03.794620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:03.796284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:03.839254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:03.839956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:03.963872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 897/1798] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:48:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:03.999612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92643840-c8f5-40dd-88c3-033cfaa43737 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:04.001807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92643840-c8f5-40dd-88c3-033cfaa43737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:04.023961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92643840-c8f5-40dd-88c3-033cfaa43737 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:04.033988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:04.035143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:04.035444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:04.035728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:04.045948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:04.047957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:04.060454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 902/1799] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:04.230419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:04.535880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:04.539103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:04.598798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6ffe4-0082-4d78-98ca-a76a989a4d13 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:04.619948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:04.643259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:04.643259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:04.643259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] PATH_INFO: `/v3/projects/345b7d8c3b9e4286b6d6d35f9cff572a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:04.643259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=345b7d8c3b9e4286b6d6d35f9cff572a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:04.643259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:04.857781 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345b7d8c3b9e4286b6d6d35f9cff572a', 'request_id': 'req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:04.868551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345b7d8c3b9e4286b6d6d35f9cff572a', 'request_id': 'req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.004193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345b7d8c3b9e4286b6d6d35f9cff572a', 'request_id': 'req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.035660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] The token cache is being invalidate because project 345b7d8c3b9e4286b6d6d35f9cff572a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:05.038194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.046939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.051387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 898/1800] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:04 2022] DELETE /identity/v3/projects/345b7d8c3b9e4286b6d6d35f9cff572a => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:05.052707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:05.066557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab4ee3-b7bd-4881-a8b4-a44dad934adf service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:05.162548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:05.163869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:05.164366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:05.165007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] PATH_INFO: `/v3/users/e056cce76b4947e5b606a0741b6121a7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:05.166596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e056cce76b4947e5b606a0741b6121a7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:05.169284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:05.603092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e056cce76b4947e5b606a0741b6121a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.621558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:05.623094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e056cce76b4947e5b606a0741b6121a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.801708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e056cce76b4947e5b606a0741b6121a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.802370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e056cce76b4947e5b606a0741b6121a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.803067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e056cce76b4947e5b606a0741b6121a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:05.834070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 903/1801] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:05 2022] DELETE /identity/v3/users/e056cce76b4947e5b606a0741b6121a7 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:06.711231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:06.714085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:06.767371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ba8fe-73dd-41a3-aa9f-b5f8c7616073 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:06.926324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:06.929311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:06.930235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:06.930613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] PATH_INFO: `/v3/projects/ef0c65bbe263445d93ba34b837cde3be` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:06.933107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0c65bbe263445d93ba34b837cde3be)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:06.937922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:07.177099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0c65bbe263445d93ba34b837cde3be', 'request_id': 'req-4ef836e8-51f5-40ab-b8c0-e39ec735210c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:07.206087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0c65bbe263445d93ba34b837cde3be', 'request_id': 'req-4ef836e8-51f5-40ab-b8c0-e39ec735210c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:07.320761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0c65bbe263445d93ba34b837cde3be', 'request_id': 'req-4ef836e8-51f5-40ab-b8c0-e39ec735210c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:07.344598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] The token cache is being invalidate because project ef0c65bbe263445d93ba34b837cde3be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:07.345164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:07.347241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:07.351332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 899/1802] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:06 2022] DELETE /identity/v3/projects/ef0c65bbe263445d93ba34b837cde3be => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:07.357224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:07.360847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:07.381844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39076d1-9f1d-491b-b5b3-f550b9d706b5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:07.511486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:07.512452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:07.512772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:07.513115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] PATH_INFO: `/v3/users/9173677075c84c97bfe993bacd35837d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:07.514670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9173677075c84c97bfe993bacd35837d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:07.517069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:07.855404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9173677075c84c97bfe993bacd35837d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:07.880213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9173677075c84c97bfe993bacd35837d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:08.009538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9173677075c84c97bfe993bacd35837d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:08.010606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9173677075c84c97bfe993bacd35837d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:08.011541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f4e08ea-b16d-4921-b2fc-5de197c09812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9173677075c84c97bfe993bacd35837d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:08.044949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 904/1803] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:07 2022] DELETE /identity/v3/users/9173677075c84c97bfe993bacd35837d => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:08.818995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:08.820971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:08.859851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef836e8-51f5-40ab-b8c0-e39ec735210c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:08.919363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:09.005385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:09.005385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:09.005385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:09.005385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] PATH_INFO: `/v3/projects/63a9280dd3e14777a2ad38891a1096d2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:09.005647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a9280dd3e14777a2ad38891a1096d2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:09.005647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:09.227883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a9280dd3e14777a2ad38891a1096d2', 'request_id': 'req-f2034f97-ecac-468e-b28e-38cbeb2ba902'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:09.242573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a9280dd3e14777a2ad38891a1096d2', 'request_id': 'req-f2034f97-ecac-468e-b28e-38cbeb2ba902'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:09.324929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a9280dd3e14777a2ad38891a1096d2', 'request_id': 'req-f2034f97-ecac-468e-b28e-38cbeb2ba902'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:09.340767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] The token cache is being invalidate because project 63a9280dd3e14777a2ad38891a1096d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:09.341111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:09.341881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f2034f97-ecac-468e-b28e-38cbeb2ba902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:09.350572 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 900/1804] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:08 2022] DELETE /identity/v3/projects/63a9280dd3e14777a2ad38891a1096d2 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:09.352492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422313d-60b6-4ff1-af98-f5cb8ee04dc0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:09.353384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422313d-60b6-4ff1-af98-f5cb8ee04dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:09.353384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422313d-60b6-4ff1-af98-f5cb8ee04dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:09.376157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2422313d-60b6-4ff1-af98-f5cb8ee04dc0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:09.378556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2422313d-60b6-4ff1-af98-f5cb8ee04dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:09.582011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:09.604662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 905/1805] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:09.607614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd934418-1535-4c13-9889-e7ed5b69f255 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:09.608673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd934418-1535-4c13-9889-e7ed5b69f255 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:09.609217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd934418-1535-4c13-9889-e7ed5b69f255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:09.649082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fd934418-1535-4c13-9889-e7ed5b69f255 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:09.651392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd934418-1535-4c13-9889-e7ed5b69f255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:09.671858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 901/1806] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:09.672471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422313d-60b6-4ff1-af98-f5cb8ee04dc0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:09.697529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422313d-60b6-4ff1-af98-f5cb8ee04dc0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:09.708460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:09.708460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:09.708460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:09.709961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:09.711840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:09.713061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:09.717087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:09.755980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 906/1807] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:09.758319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd934418-1535-4c13-9889-e7ed5b69f255 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:09.779803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd934418-1535-4c13-9889-e7ed5b69f255 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:09.786712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:09.787788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:09.788076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:09.788355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:09.789167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:09.791307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:09.903796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 902/1808] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:09 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:09.910807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:09.935765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe4de-4224-4165-a1e3-a9b2b7048f03 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:10.082279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:10.083000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:10.083462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:10.083766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:10.084701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:10.106536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:10.281781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 907/1809] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:09 2022] POST /identity/v3/users => generated 291 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:10.283954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:10.311821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd55a1-b985-4f1d-b2db-449f2f0c4c34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:10.323730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:10.325469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:10.325974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:10.326458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:10.327789 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:10.330920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:10.353835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 903/1810] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:10.358306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:10.379482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26218455-d0d6-47b1-98f2-a61685ec03b9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:10.389817 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab831d4-fca0-4a5b-84c0-14c758735ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:10.391815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab831d4-fca0-4a5b-84c0-14c758735ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:10.394183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab831d4-fca0-4a5b-84c0-14c758735ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:10.394941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab831d4-fca0-4a5b-84c0-14c758735ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:10.397277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab831d4-fca0-4a5b-84c0-14c758735ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:10.400913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab831d4-fca0-4a5b-84c0-14c758735ba2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:10.436722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 908/1811] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:10.452316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:10.481508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2c39-6655-42e0-b448-ffaffbff7a17 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:10.511516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:10.517390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:10.518926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:10.519609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] PATH_INFO: `/v3/projects/cc93b65a84d143aa933e84e7cde9e166/users/3e1a889cb5cd4c20a43b8e7e2fff93b5/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:10.582845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc93b65a84d143aa933e84e7cde9e166, user_id=3e1a889cb5cd4c20a43b8e7e2fff93b5, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:10.586159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:10.764392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 904/1812] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:10 2022] PUT /identity/v3/projects/cc93b65a84d143aa933e84e7cde9e166/users/3e1a889cb5cd4c20a43b8e7e2fff93b5/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:10.780485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ef82e-1dd3-48f4-8ec4-6d0a93c8507e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:10.780973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ef82e-1dd3-48f4-8ec4-6d0a93c8507e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:10.781442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ef82e-1dd3-48f4-8ec4-6d0a93c8507e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:10.858229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-465ef82e-1dd3-48f4-8ec4-6d0a93c8507e None None] MFA Rules not processed for user `3e1a889cb5cd4c20a43b8e7e2fff93b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:10.862601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-465ef82e-1dd3-48f4-8ec4-6d0a93c8507e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:11.050897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 909/1813] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:11.052527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:11.147655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c16d9-7437-4529-ae53-961bad1ff035 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:11.153407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:11.154147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:11.154529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:11.154897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:11.155899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:11.159495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:11.309580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 905/1814] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:11 2022] POST /identity/v3/projects => generated 381 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:11.312983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ef82e-1dd3-48f4-8ec4-6d0a93c8507e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:11.327292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ef82e-1dd3-48f4-8ec4-6d0a93c8507e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:11.487177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:11.499209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:11.499522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:11.499811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:11.511280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:11.512361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:11.715486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 910/1815] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:11 2022] POST /identity/v3/users => generated 297 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:11.723828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:11.741345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6dbb-4fa8-4474-a466-651dcdbb44ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:11.757487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:11.758283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:11.758616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:11.758981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:11.759786 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:11.761838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:11.776695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 906/1816] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:11.784193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:11.817760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d7b2d-9138-4eaf-aa6d-c352de12ecbe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:11.823865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:11.824968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:11.825332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:11.825729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] PATH_INFO: `/v3/projects/dd9919a1830d45299cad19b06c5da9f1/users/cdb5ecccc9bc4a438aa69cbb02af7de2/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:11.844286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9919a1830d45299cad19b06c5da9f1, user_id=cdb5ecccc9bc4a438aa69cbb02af7de2, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:11.848747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:11.990483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 911/1817] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:11 2022] PUT /identity/v3/projects/dd9919a1830d45299cad19b06c5da9f1/users/cdb5ecccc9bc4a438aa69cbb02af7de2/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:11.994887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:12.011784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509178a-b826-43d5-9c6d-4fb37e5665b0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:12.139429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa942198-6fcc-4284-8438-ee1ef743801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:12.140436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa942198-6fcc-4284-8438-ee1ef743801e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:12.140436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa942198-6fcc-4284-8438-ee1ef743801e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:12.140707 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa942198-6fcc-4284-8438-ee1ef743801e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:12.141480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa942198-6fcc-4284-8438-ee1ef743801e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:12.143400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa942198-6fcc-4284-8438-ee1ef743801e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:12.155574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 907/1818] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:12.164424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:12.193702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93805723-553a-4576-a923-a7f6b0634231 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:12.202598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:12.205337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:12.208645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:12.208645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] PATH_INFO: `/v3/projects/dd9919a1830d45299cad19b06c5da9f1/users/cdb5ecccc9bc4a438aa69cbb02af7de2/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:12.215398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9919a1830d45299cad19b06c5da9f1, user_id=cdb5ecccc9bc4a438aa69cbb02af7de2, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:12.223176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:12.244607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 912/1819] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:12 2022] PUT /identity/v3/projects/dd9919a1830d45299cad19b06c5da9f1/users/cdb5ecccc9bc4a438aa69cbb02af7de2/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:12.255961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc909c69-b789-4100-9a24-bdc9c973a0f3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:12.256298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc909c69-b789-4100-9a24-bdc9c973a0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:12.256506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc909c69-b789-4100-9a24-bdc9c973a0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:12.302544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bc909c69-b789-4100-9a24-bdc9c973a0f3 None None] MFA Rules not processed for user `cdb5ecccc9bc4a438aa69cbb02af7de2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:12.305476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc909c69-b789-4100-9a24-bdc9c973a0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:12.413433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 908/1820] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:12.431338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:12.436393 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:12.521720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b870d-075a-4f63-bdcc-8948e15e1785 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:12.526037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b4d10-8abb-47a8-8a1d-3ec2a1d8958c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:12.529099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b4d10-8abb-47a8-8a1d-3ec2a1d8958c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:12.529468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b4d10-8abb-47a8-8a1d-3ec2a1d8958c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:12.530053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b4d10-8abb-47a8-8a1d-3ec2a1d8958c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:12.549669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b4d10-8abb-47a8-8a1d-3ec2a1d8958c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:12.550633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b4d10-8abb-47a8-8a1d-3ec2a1d8958c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:12.574966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 913/1821] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:12.734611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc909c69-b789-4100-9a24-bdc9c973a0f3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:12.748625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc909c69-b789-4100-9a24-bdc9c973a0f3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:12.754996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981286b5-cd55-4d66-a22d-efe120978c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:12.755858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981286b5-cd55-4d66-a22d-efe120978c4e service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:12.756760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981286b5-cd55-4d66-a22d-efe120978c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:12.757079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981286b5-cd55-4d66-a22d-efe120978c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:12.848865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981286b5-cd55-4d66-a22d-efe120978c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:12.851376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981286b5-cd55-4d66-a22d-efe120978c4e service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:12.865892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 909/1822] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:13.272919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:14.420021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:14.667980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f45d73-340d-40b2-9bdd-3aa4fc641938 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:14.668270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f45d73-340d-40b2-9bdd-3aa4fc641938 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:14.668723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f45d73-340d-40b2-9bdd-3aa4fc641938 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:14.670472 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 914/1823] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:48:14.687972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:14.690573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:14.690895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:14.714787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] MFA Rules not processed for user `8defe1e0d31249aa8bb27610229781b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:14.716486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:14.819021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 910/1824] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:48:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:15.004649 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f45d73-340d-40b2-9bdd-3aa4fc641938 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:15.007519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11f45d73-340d-40b2-9bdd-3aa4fc641938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:15.030357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f45d73-340d-40b2-9bdd-3aa4fc641938 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:15.047830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:15.049261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:15.049684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:15.050081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:15.060106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:15.062204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:15.075047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 915/1825] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:16.509360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:16.510896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:16.602193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba367a78-de86-40c0-a4b2-ecbffb48c2f4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:16.606947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:16.607865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:16.608210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:16.608494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] PATH_INFO: `/v3/users/c597f753e1b542b794c3c68d9cbba8b7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:16.609742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c597f753e1b542b794c3c68d9cbba8b7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:16.611738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:16.916680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c597f753e1b542b794c3c68d9cbba8b7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:16.917772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:16.926587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c597f753e1b542b794c3c68d9cbba8b7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:16.930931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2583875-c1ac-44c3-aef7-6c624dae97d7 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:16.939907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:16.941172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:16.941544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:16.941823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] PATH_INFO: `/v3/users/3e1a889cb5cd4c20a43b8e7e2fff93b5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:16.946326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e1a889cb5cd4c20a43b8e7e2fff93b5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:16.946459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c597f753e1b542b794c3c68d9cbba8b7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:16.946567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:17.100103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c597f753e1b542b794c3c68d9cbba8b7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:17.100893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-072c5364-0f72-4fb5-940c-99a78b24898f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c597f753e1b542b794c3c68d9cbba8b7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:17.110431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 911/1826] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:16 2022] DELETE /identity/v3/users/c597f753e1b542b794c3c68d9cbba8b7 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:17.142898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:17.217641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e1a889cb5cd4c20a43b8e7e2fff93b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:17.240340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e1a889cb5cd4c20a43b8e7e2fff93b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:17.317647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e1a889cb5cd4c20a43b8e7e2fff93b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:17.318815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e1a889cb5cd4c20a43b8e7e2fff93b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:17.319972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e1a889cb5cd4c20a43b8e7e2fff93b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:17.337396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 916/1827] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:16 2022] DELETE /identity/v3/users/3e1a889cb5cd4c20a43b8e7e2fff93b5 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:17.341935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:17.342432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:17.342821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:17.381954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:17.384761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:17.490448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 912/1828] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:17.503608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:17.505965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:17.522789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:17.592611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc0bb-daa7-42f4-9133-df06419c5149 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:17.600023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:17.601218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:17.601766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:17.602273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:17.621283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:17.624127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:17.639068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 917/1829] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:18.307941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:18.310108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:18.329007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018bb28-f6cc-44f1-a3c7-ad6c10aaf6dd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:18.336578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:18.338014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:18.338505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:18.339013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] PATH_INFO: `/v3/projects/cc93b65a84d143aa933e84e7cde9e166` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:18.347506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc93b65a84d143aa933e84e7cde9e166)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:18.351506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:18.415634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:18.417485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:18.438341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72288ed4-b2d3-4b04-a7d6-f3ed93b22943 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:18.445306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:18.446316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:18.446728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:18.447111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] PATH_INFO: `/v3/projects/6cd30e8c51cf4665b4c5f5262b56053f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:18.448992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd30e8c51cf4665b4c5f5262b56053f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:18.451208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:18.526366 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc93b65a84d143aa933e84e7cde9e166', 'request_id': 'req-afe39187-b775-4cbe-b7e5-e6b61461b900'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.541832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc93b65a84d143aa933e84e7cde9e166', 'request_id': 'req-afe39187-b775-4cbe-b7e5-e6b61461b900'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.661821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd30e8c51cf4665b4c5f5262b56053f', 'request_id': 'req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.675141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc93b65a84d143aa933e84e7cde9e166', 'request_id': 'req-afe39187-b775-4cbe-b7e5-e6b61461b900'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.691335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] The token cache is being invalidate because project cc93b65a84d143aa933e84e7cde9e166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:18.691335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.691335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.693161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 913/1830] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:18 2022] DELETE /identity/v3/projects/cc93b65a84d143aa933e84e7cde9e166 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:18.697053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:18.701681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:18.788448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe39187-b775-4cbe-b7e5-e6b61461b900 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:18.851356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd30e8c51cf4665b4c5f5262b56053f', 'request_id': 'req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.866520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd30e8c51cf4665b4c5f5262b56053f', 'request_id': 'req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.890875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] The token cache is being invalidate because project 6cd30e8c51cf4665b4c5f5262b56053f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:18.891394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.893088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:18.900947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:18.900947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:18.901354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:18.901745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] PATH_INFO: `/v3/users/cdb5ecccc9bc4a438aa69cbb02af7de2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:18.901969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 918/1831] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:18 2022] DELETE /identity/v3/projects/6cd30e8c51cf4665b4c5f5262b56053f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:18.903179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb5ecccc9bc4a438aa69cbb02af7de2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:18.904079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:18.905574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:18.906916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:18.930629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3e9d7-aa2c-47ef-a5ae-86f539193b3e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:18.935784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:18.936887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:18.937169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:18.937449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] PATH_INFO: `/v3/users/ea9272ca9692490cae3d889676b61ba9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:18.939110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9272ca9692490cae3d889676b61ba9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:18.943988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:19.244272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb5ecccc9bc4a438aa69cbb02af7de2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.273615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb5ecccc9bc4a438aa69cbb02af7de2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.304391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb5ecccc9bc4a438aa69cbb02af7de2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.338403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9272ca9692490cae3d889676b61ba9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.355559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9272ca9692490cae3d889676b61ba9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.432017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb5ecccc9bc4a438aa69cbb02af7de2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.448305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb5ecccc9bc4a438aa69cbb02af7de2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.456963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 914/1832] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:18 2022] DELETE /identity/v3/users/cdb5ecccc9bc4a438aa69cbb02af7de2 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:19.729642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9272ca9692490cae3d889676b61ba9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.732162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9272ca9692490cae3d889676b61ba9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.732922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9272ca9692490cae3d889676b61ba9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:19.760890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 919/1833] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:18 2022] DELETE /identity/v3/users/ea9272ca9692490cae3d889676b61ba9 => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:20.176230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:20.178971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:20.310031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:20.340277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791f46a-b52c-49bc-b478-aaae911a3c4e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:20.442334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:20.445231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:20.446161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:20.446448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:20.446765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] PATH_INFO: `/v3/users/a52fecde74244b009bf48d41c8049f84` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:20.446953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:20.463049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52fecde74244b009bf48d41c8049f84)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:20.467607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:20.509951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6b480-3b70-41e9-b500-35929cc4b038 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:20.523548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:20.526936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:20.527835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:20.533950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] PATH_INFO: `/v3/projects/811a92f9ef894cfc8e2de198435000e4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:20.533950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] RBAC: Authorizing `identity:delete_project(project_id=811a92f9ef894cfc8e2de198435000e4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:20.537693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:20.884562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811a92f9ef894cfc8e2de198435000e4', 'request_id': 'req-71e621f7-9a2c-45af-8cd2-186d731eb38e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.885834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52fecde74244b009bf48d41c8049f84'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.903051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52fecde74244b009bf48d41c8049f84'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.931696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52fecde74244b009bf48d41c8049f84'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.973966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811a92f9ef894cfc8e2de198435000e4', 'request_id': 'req-71e621f7-9a2c-45af-8cd2-186d731eb38e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.982139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811a92f9ef894cfc8e2de198435000e4', 'request_id': 'req-71e621f7-9a2c-45af-8cd2-186d731eb38e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.996090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] The token cache is being invalidate because project 811a92f9ef894cfc8e2de198435000e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:20.996530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.997282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:20.999024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 920/1834] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:20 2022] DELETE /identity/v3/projects/811a92f9ef894cfc8e2de198435000e4 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:21.003591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.003591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.066821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52fecde74244b009bf48d41c8049f84'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:21.072111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7b01614a-63ad-44ce-b5a8-524b78eae121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52fecde74244b009bf48d41c8049f84'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:21.077005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 915/1835] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:20 2022] DELETE /identity/v3/users/a52fecde74244b009bf48d41c8049f84 => generated 0 bytes in 902 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:21.078992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa2fbf-c2c8-450d-b76b-942dbe696982 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.080286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa2fbf-c2c8-450d-b76b-942dbe696982 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.080680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa2fbf-c2c8-450d-b76b-942dbe696982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.099567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e621f7-9a2c-45af-8cd2-186d731eb38e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.113830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2eaa2fbf-c2c8-450d-b76b-942dbe696982 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:21.115200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2eaa2fbf-c2c8-450d-b76b-942dbe696982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.188823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.189796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.203489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.203489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] PATH_INFO: `/v3/projects/dd9919a1830d45299cad19b06c5da9f1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.203489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9919a1830d45299cad19b06c5da9f1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.203489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.203829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 916/1836] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:21.203829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29ee1-2364-4bc7-9e9a-d06961008c68 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.203829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29ee1-2364-4bc7-9e9a-d06961008c68 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.203829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29ee1-2364-4bc7-9e9a-d06961008c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.243703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-04c29ee1-2364-4bc7-9e9a-d06961008c68 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:21.247071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-04c29ee1-2364-4bc7-9e9a-d06961008c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.262093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 917/1837] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:21.262093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.262499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.262499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.296093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:21.297496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.307333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 918/1838] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:21.309355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.311639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.349666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9919a1830d45299cad19b06c5da9f1', 'request_id': 'req-b1f22a84-0dc6-440b-98ad-446a76383734'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:21.393620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2844a-bd1d-4118-bc16-b14bf99b230f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.443453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9919a1830d45299cad19b06c5da9f1', 'request_id': 'req-b1f22a84-0dc6-440b-98ad-446a76383734'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:21.451262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9919a1830d45299cad19b06c5da9f1', 'request_id': 'req-b1f22a84-0dc6-440b-98ad-446a76383734'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:21.456034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f79f6-e9fc-4a4c-8482-c5a8229ea4a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.456843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f79f6-e9fc-4a4c-8482-c5a8229ea4a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.457270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f79f6-e9fc-4a4c-8482-c5a8229ea4a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.457577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f79f6-e9fc-4a4c-8482-c5a8229ea4a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.465214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] The token cache is being invalidate because project dd9919a1830d45299cad19b06c5da9f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:21.465839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:21.466780 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:21.471747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 921/1839] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:20 2022] DELETE /identity/v3/projects/dd9919a1830d45299cad19b06c5da9f1 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:21.473043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.474831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.550589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f79f6-e9fc-4a4c-8482-c5a8229ea4a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.553023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f79f6-e9fc-4a4c-8482-c5a8229ea4a9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.570161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22a84-0dc6-440b-98ad-446a76383734 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.571831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 919/1840] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:21.573833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62831-fff1-401d-b38b-baf907192977 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.574321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62831-fff1-401d-b38b-baf907192977 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.574870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62831-fff1-401d-b38b-baf907192977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.576492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.577507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.578013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.581190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.581190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:21.581190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.582444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.596936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 922/1841] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:21.601881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.604395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-65a62831-fff1-401d-b38b-baf907192977 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:21.606641 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-65a62831-fff1-401d-b38b-baf907192977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.615062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef6081-4b16-4539-adf1-e37502fcc4a1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.621532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.622637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.623169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.623729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.624526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 920/1842] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:21.624883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.627910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.635523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67cb8-cef5-4c17-a963-8c05ca5cdbec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.637179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67cb8-cef5-4c17-a963-8c05ca5cdbec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.638573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67cb8-cef5-4c17-a963-8c05ca5cdbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.676215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-03d67cb8-cef5-4c17-a963-8c05ca5cdbec None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:21.678677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03d67cb8-cef5-4c17-a963-8c05ca5cdbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.695498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 921/1843] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:21.701794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67cb8-cef5-4c17-a963-8c05ca5cdbec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.717006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67cb8-cef5-4c17-a963-8c05ca5cdbec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.726465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.727395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.727784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.728719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.729172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:21.729584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.731740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.760888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 922/1844] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:48:21.764606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 923/1845] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:21.767589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.770678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.793489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b949-9a54-4d7f-b25e-45e563d15455 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.887184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.888665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.889124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.889550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.890876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.892651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc717c1-0290-49f4-b9b6-fdc6bc719fb8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.894429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.898690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.899725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.900035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.900413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.902506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.904914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.987694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:22.068243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 923/1846] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/projects => generated 359 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:22.072994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.136434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 924/1847] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:21 2022] POST /identity/v3/users => generated 293 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:22.137490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.150828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bff34-97ae-47bc-acd0-15348f7f6a2e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.197781 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd15d-4dd6-470f-950d-209787a8a829 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.208118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.209024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.209393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.209773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.210732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.213096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.286463 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.287315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.287599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.287878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.288620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.290301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.302637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 925/1848] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:22.303522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.305284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:22.319029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2375fc-6997-407d-96c8-d997b09cd67c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.324116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.325087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.325371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.325652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] PATH_INFO: `/v3/projects/d683c9972fdd4996bf6b82aa16f1b50a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.327199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d683c9972fdd4996bf6b82aa16f1b50a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.329213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.335973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 924/1849] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:22 2022] POST /identity/v3/users => generated 280 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:22.337154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.346049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:22.353404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c9c20-1767-4df8-9403-79722a944124 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.359471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.360241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.360696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.361922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.363042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.364747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.377661 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 925/1850] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:22.378214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.391438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb20fca-2b8c-4692-85a2-26e84bede0e7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.399457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.400168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.400457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.400738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.401658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.403340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.419205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 926/1851] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:22.420109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.432299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d0f7-f82a-401e-a6c2-b85e063c1e35 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.437746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.438667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.439086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.440194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] PATH_INFO: `/v3/projects/701ef2c2ee604a2bbde59d1f9429a352/users/fc85911a338e412e882c63d44705d5b3/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.457814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701ef2c2ee604a2bbde59d1f9429a352, user_id=fc85911a338e412e882c63d44705d5b3, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.464054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.483494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd683c9972fdd4996bf6b82aa16f1b50a', 'request_id': 'req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.587848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 927/1852] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:22 2022] PUT /identity/v3/projects/701ef2c2ee604a2bbde59d1f9429a352/users/fc85911a338e412e882c63d44705d5b3/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:22.588699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.641498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd683c9972fdd4996bf6b82aa16f1b50a', 'request_id': 'req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.648182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd683c9972fdd4996bf6b82aa16f1b50a', 'request_id': 'req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.655499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac8e6-9ba0-41d2-afa3-a7a875543087 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.660599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] The token cache is being invalidate because project d683c9972fdd4996bf6b82aa16f1b50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:22.660931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.661504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.661635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d8ff4558-b4bd-4ef8-9eff-78633a6b4321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.662227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.662483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.662762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.663220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 926/1853] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:22 2022] DELETE /identity/v3/projects/d683c9972fdd4996bf6b82aa16f1b50a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:22.663496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.664617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.664937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.665181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.665273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.677766 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 928/1854] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:22.678639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.680524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:22.695812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] MFA Rules not processed for user `fc85911a338e412e882c63d44705d5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:22.696227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f4af8-b087-4a36-8719-c9ea76820ac1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.697056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:22.701042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.701941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.702223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.702491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] PATH_INFO: `/v3/users/0c85ae8409f24d3883c5c0a48a492956` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.704121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c85ae8409f24d3883c5c0a48a492956)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.706725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.770003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 927/1855] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:22.770954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.772239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:22.786540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f24b11-49af-4541-80d1-c409bb8e9255 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:22.791363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:22.792291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:22.792573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:22.792879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] PATH_INFO: `/v3/projects/77af88872a914445bca5a39882f9a2ae/users/fea9886eb4cc479bbb4be23168fed637/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:22.816112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77af88872a914445bca5a39882f9a2ae, user_id=fea9886eb4cc479bbb4be23168fed637, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:22.818955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:22.938461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 928/1856] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:22 2022] PUT /identity/v3/projects/77af88872a914445bca5a39882f9a2ae/users/fea9886eb4cc479bbb4be23168fed637/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:22.939569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:22.941378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:22.969774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c85ae8409f24d3883c5c0a48a492956'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.978836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c85ae8409f24d3883c5c0a48a492956'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.991375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c85ae8409f24d3883c5c0a48a492956'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:22.999812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a69e5a-d454-4bc6-bac8-c7ff9d4518e9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:23.003510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf362cf-1982-453b-9601-c66c7925b52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:23.004161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf362cf-1982-453b-9601-c66c7925b52a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.004447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf362cf-1982-453b-9601-c66c7925b52a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.004727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf362cf-1982-453b-9601-c66c7925b52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.062812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf362cf-1982-453b-9601-c66c7925b52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:23.065244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf362cf-1982-453b-9601-c66c7925b52a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:23.074120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 929/1857] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:23.075406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.075688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.075967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.098191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c85ae8409f24d3883c5c0a48a492956'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.098887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c85ae8409f24d3883c5c0a48a492956'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.105644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 929/1858] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:22 2022] DELETE /identity/v3/users/0c85ae8409f24d3883c5c0a48a492956 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:23.106798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:23.106901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] MFA Rules not processed for user `fea9886eb4cc479bbb4be23168fed637`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:23.108199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:23.108990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:23.189308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a621-7e5b-4289-a180-f27a38a43dda admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:23.194534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:23.203413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.203413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.203413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] PATH_INFO: `/v3/users/fc85911a338e412e882c63d44705d5b3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.203413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc85911a338e412e882c63d44705d5b3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:23.203413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:23.205942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 930/1859] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:23 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:23.241870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:23.243409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:23.308622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcd8d2-d4fb-4070-9611-4236634b79c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:23.312647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:23.313451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.313839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.314236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.326592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:23.328592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:23.353518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 931/1860] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:23 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:23.370925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:23.372829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:23.398148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc44d88-9f93-4bd8-bdd0-4ce08a55f74c service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:23.405628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:23.406856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.407421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.408425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] PATH_INFO: `/v3/users/fea9886eb4cc479bbb4be23168fed637` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.410376 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea9886eb4cc479bbb4be23168fed637)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:23.413229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:23.492493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc85911a338e412e882c63d44705d5b3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.501779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc85911a338e412e882c63d44705d5b3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.518148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc85911a338e412e882c63d44705d5b3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.612730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc85911a338e412e882c63d44705d5b3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.613547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-afa50eeb-f1d9-4bb3-b144-095d0b09596a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc85911a338e412e882c63d44705d5b3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.620849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 930/1861] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:23 2022] DELETE /identity/v3/users/fc85911a338e412e882c63d44705d5b3 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:23.628594 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.629142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.629687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.653032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:23.654348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:23.733004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 931/1862] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:23.734128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:23.735466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:23.744237 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea9886eb4cc479bbb4be23168fed637'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.754411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4210a-40cd-41e8-9ad9-0f9d5ed2a62b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:23.759359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea9886eb4cc479bbb4be23168fed637'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.761621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:23.762535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.762821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.763147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] PATH_INFO: `/v3/projects/054875c2a7cc4838bd05084826ba0683` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.764382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=054875c2a7cc4838bd05084826ba0683)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:23.766502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:23.837637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea9886eb4cc479bbb4be23168fed637'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.838297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea9886eb4cc479bbb4be23168fed637'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.839495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea9886eb4cc479bbb4be23168fed637'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.853857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 932/1863] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:23 2022] DELETE /identity/v3/users/fea9886eb4cc479bbb4be23168fed637 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:23.854669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:23.855977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:23.919304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414427d-505f-4af6-a71a-48f185cb98c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:23.921460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '054875c2a7cc4838bd05084826ba0683', 'request_id': 'req-2615735f-083e-4e66-ac32-3b2351df7457'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.929851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '054875c2a7cc4838bd05084826ba0683', 'request_id': 'req-2615735f-083e-4e66-ac32-3b2351df7457'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:23.984861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66e4bb-7911-4f07-a6e2-60d62aa80a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:23.985429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66e4bb-7911-4f07-a6e2-60d62aa80a68 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:23.985707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66e4bb-7911-4f07-a6e2-60d62aa80a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:23.986549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66e4bb-7911-4f07-a6e2-60d62aa80a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:23.988749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb66e4bb-7911-4f07-a6e2-60d62aa80a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:24.020100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '054875c2a7cc4838bd05084826ba0683', 'request_id': 'req-2615735f-083e-4e66-ac32-3b2351df7457'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:24.033998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] The token cache is being invalidate because project 054875c2a7cc4838bd05084826ba0683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:24.034405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:24.035015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2615735f-083e-4e66-ac32-3b2351df7457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:24.036681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 932/1864] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:23 2022] DELETE /identity/v3/projects/054875c2a7cc4838bd05084826ba0683 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:24.037984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.038365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.038565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.056123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66e4bb-7911-4f07-a6e2-60d62aa80a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:24.057634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66e4bb-7911-4f07-a6e2-60d62aa80a68 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:24.062727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:24.064032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:24.072352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 933/1865] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:24.072517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 933/1866] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:24.074591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4719c-e642-4591-8459-54949cbe149e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.075107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4719c-e642-4591-8459-54949cbe149e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.077146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4719c-e642-4591-8459-54949cbe149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.085691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:24.088772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:24.099370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-71c4719c-e642-4591-8459-54949cbe149e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:24.100601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71c4719c-e642-4591-8459-54949cbe149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:24.110488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 934/1867] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:24.112679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35913d-3810-43eb-bc51-0b6f4403b3a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:24.121794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1aa02-f21b-49b8-9e04-6e9d8994f373 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.121989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:24.123033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.123561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.123703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1aa02-f21b-49b8-9e04-6e9d8994f373 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.123978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.124099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1aa02-f21b-49b8-9e04-6e9d8994f373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.139422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:24.141885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:24.150954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4aa1aa02-f21b-49b8-9e04-6e9d8994f373 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:24.152571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4aa1aa02-f21b-49b8-9e04-6e9d8994f373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:24.162141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 934/1868] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:24.166734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 935/1869] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:24.169873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:24.182754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c92809-3f1f-48f0-bd1d-1f6aab1f4788 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:24.187793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:24.188380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.188726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.189263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.189981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:24.190293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:24.192023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:24.204226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 935/1870] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:24.209583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1aa02-f21b-49b8-9e04-6e9d8994f373 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:24.219454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1aa02-f21b-49b8-9e04-6e9d8994f373 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:24.225818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:24.226466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.226748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.227024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.227859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:24.229935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:24.395489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 936/1871] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:24 2022] POST /identity/v3/projects => generated 393 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:24.399546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:24.411116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7330c-83b2-4a7a-8680-d0ceafeb621a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:24.508870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:24.508870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.508870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.508870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.509716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:24.509716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:24.612259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:24.725161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:24.728249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:24.749993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba13026-c5ae-4d43-9dc2-4d7e459250cf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:24.757460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:24.758575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.759046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.759559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] PATH_INFO: `/v3/projects/701ef2c2ee604a2bbde59d1f9429a352` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.761090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 936/1872] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:24 2022] POST /identity/v3/users => generated 297 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:24.761710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=701ef2c2ee604a2bbde59d1f9429a352)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:24.762212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:24.766057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:24.772576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:24.814689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3802d-48a9-438c-87f7-a55a78fe7c0d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:24.820007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:24.821117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:24.821409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:24.822061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] PATH_INFO: `/v3/projects/77af88872a914445bca5a39882f9a2ae` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:24.823703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77af88872a914445bca5a39882f9a2ae)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:24.826092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:24.910468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701ef2c2ee604a2bbde59d1f9429a352', 'request_id': 'req-554a26f5-aabf-469a-b1f4-d3229001d000'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.018120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77af88872a914445bca5a39882f9a2ae', 'request_id': 'req-825e21d1-bbf5-47fa-8577-b67b42c01248'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.025914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77af88872a914445bca5a39882f9a2ae', 'request_id': 'req-825e21d1-bbf5-47fa-8577-b67b42c01248'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.032522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701ef2c2ee604a2bbde59d1f9429a352', 'request_id': 'req-554a26f5-aabf-469a-b1f4-d3229001d000'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.035504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:25.042927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701ef2c2ee604a2bbde59d1f9429a352', 'request_id': 'req-554a26f5-aabf-469a-b1f4-d3229001d000'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.057357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] The token cache is being invalidate because project 701ef2c2ee604a2bbde59d1f9429a352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:25.057888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.058730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.060326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 937/1873] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:24 2022] DELETE /identity/v3/projects/701ef2c2ee604a2bbde59d1f9429a352 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:25.064289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.064289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:25.148047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a26f5-aabf-469a-b1f4-d3229001d000 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.157575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77af88872a914445bca5a39882f9a2ae', 'request_id': 'req-825e21d1-bbf5-47fa-8577-b67b42c01248'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.170846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] The token cache is being invalidate because project 77af88872a914445bca5a39882f9a2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:25.171146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.171825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-825e21d1-bbf5-47fa-8577-b67b42c01248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:25.174083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 937/1874] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:24 2022] DELETE /identity/v3/projects/77af88872a914445bca5a39882f9a2ae => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:25.175389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094157-4f53-4c2f-84e0-1f54645fc98e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.175681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094157-4f53-4c2f-84e0-1f54645fc98e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.175959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094157-4f53-4c2f-84e0-1f54645fc98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.201561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5b094157-4f53-4c2f-84e0-1f54645fc98e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:25.202807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5b094157-4f53-4c2f-84e0-1f54645fc98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:25.215355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43961d55-a859-40dd-ba10-15dfcc1ca0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.216049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43961d55-a859-40dd-ba10-15dfcc1ca0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.216333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43961d55-a859-40dd-ba10-15dfcc1ca0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.216664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43961d55-a859-40dd-ba10-15dfcc1ca0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.217520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43961d55-a859-40dd-ba10-15dfcc1ca0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.219178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43961d55-a859-40dd-ba10-15dfcc1ca0b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.230837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 938/1875] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:25.235610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.236383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.236560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.256322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:25.257582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:25.266848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 939/1876] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:25.268453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.269766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:25.274017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 938/1877] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:25.275476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448859a-245e-44f6-8358-969606aafabb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.275663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448859a-245e-44f6-8358-969606aafabb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.275936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448859a-245e-44f6-8358-969606aafabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.283671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f3dde-7216-4415-b36c-7e107f7573e9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.288848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4ca6e-6dc3-4dc6-b552-11abe56cc5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.289531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4ca6e-6dc3-4dc6-b552-11abe56cc5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.289814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4ca6e-6dc3-4dc6-b552-11abe56cc5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.290091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4ca6e-6dc3-4dc6-b552-11abe56cc5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.290829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4ca6e-6dc3-4dc6-b552-11abe56cc5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.292478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4ca6e-6dc3-4dc6-b552-11abe56cc5c8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.296915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5448859a-245e-44f6-8358-969606aafabb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:25.298344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5448859a-245e-44f6-8358-969606aafabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:25.304494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 940/1878] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:25.305954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bba1ea-11ec-497b-867a-42209b8ed6dc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.306329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bba1ea-11ec-497b-867a-42209b8ed6dc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.306625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bba1ea-11ec-497b-867a-42209b8ed6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.307416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 939/1879] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:25.311963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448859a-245e-44f6-8358-969606aafabb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.323100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448859a-245e-44f6-8358-969606aafabb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.325587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b9bba1ea-11ec-497b-867a-42209b8ed6dc None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:25.327440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b9bba1ea-11ec-497b-867a-42209b8ed6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:25.329673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.330621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.331026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.332755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] PATH_INFO: `/v3/projects/eee60a7fa3854c46bbf7e85a4aa49cab/users/a41c687c751e425c93346421566687bd/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.335489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 941/1880] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:25.336342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bba1ea-11ec-497b-867a-42209b8ed6dc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.347461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bba1ea-11ec-497b-867a-42209b8ed6dc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.352505 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.353219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.353497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.353629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee60a7fa3854c46bbf7e85a4aa49cab, user_id=a41c687c751e425c93346421566687bd, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.353754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.354502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:25.354805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.356424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.356662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.370483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 942/1881] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:25.371359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.382764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2c33a-5294-4164-b95b-18ae613b1d87 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.387851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.388541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.388858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.389135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.389881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:25.390252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.392074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.405402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 943/1882] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:25.406240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.418795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefcef8-94ab-4d61-9c67-3b5d2a64daea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.424358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4075-1177-4baf-a96d-06d29725aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.425220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4075-1177-4baf-a96d-06d29725aece admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.425514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4075-1177-4baf-a96d-06d29725aece admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.425803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4075-1177-4baf-a96d-06d29725aece admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.426512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4075-1177-4baf-a96d-06d29725aece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.428465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4075-1177-4baf-a96d-06d29725aece admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.468635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 940/1883] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:25 2022] PUT /identity/v3/projects/eee60a7fa3854c46bbf7e85a4aa49cab/users/a41c687c751e425c93346421566687bd/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:25.469724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.546182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fbb0d-e624-4f10-986b-d0e1612f0f39 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.552234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.552951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.553237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.553512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.554228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.556829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.634443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 944/1884] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/projects => generated 369 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:25.635999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f81ac-3f3b-4f22-9e1d-7dd525e7d96b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.636286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f81ac-3f3b-4f22-9e1d-7dd525e7d96b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.636624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f81ac-3f3b-4f22-9e1d-7dd525e7d96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.667224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0b7f81ac-3f3b-4f22-9e1d-7dd525e7d96b None None] MFA Rules not processed for user `a41c687c751e425c93346421566687bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:25.668454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b7f81ac-3f3b-4f22-9e1d-7dd525e7d96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:25.737968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 941/1885] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/projects => generated 375 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:25.739131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.750226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 945/1886] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:25.751057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f81ac-3f3b-4f22-9e1d-7dd525e7d96b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:25.849582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f81ac-3f3b-4f22-9e1d-7dd525e7d96b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.856643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.857566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.857996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992c25-e430-4828-afdf-740c73f72272 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:25.858284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.858746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.859670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.863088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:25.864048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:25.864744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:25.865063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:25.865337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:25.866131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:25.868182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.012454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 946/1887] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:26.012454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.022271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600dea-9a84-4f78-92b7-7a2ac67f071d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.027066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.027785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.028071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.028348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.029088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.031048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.058819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 942/1888] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:25 2022] POST /identity/v3/users => generated 285 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:26.062484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.072659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b9b8-98c0-4826-aca8-560bd83ac753 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.078823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.080256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.080459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.080878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.081572 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.083228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.101180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 943/1889] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:26.102155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.115389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aefd86-8b2e-45ca-aae7-9cd6f49ffbd8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.122356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.123058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.123349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.123641 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.124395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.126128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.137882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 944/1890] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:26.139359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.151556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0f3af-0390-4bd7-b985-dae19aea64e3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.160615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.161599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.161900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.162211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.163237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.166211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.184467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 945/1891] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:26.184467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.196366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 947/1892] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:26 2022] POST /identity/v3/projects => generated 393 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:26.197108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db3e3a-cc9d-4008-ab10-955fdb0c0371 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.198008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.294779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84023-42a6-4957-b638-cbb5e3012b9c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.295688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.296512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.296940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.297419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.299223 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.301226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.301934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.303335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.303335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.303484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] PATH_INFO: `/v3/projects/26182f5095e94f26a90b9dc0d27af3ae/users/4a46b603542744eeb1aa5f75772bafd7/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.319845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 946/1893] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:26.321297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.329762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26182f5095e94f26a90b9dc0d27af3ae, user_id=4a46b603542744eeb1aa5f75772bafd7, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.334362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.337620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740cc16-3b14-4d2e-94d6-0285ea565e83 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.343747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ad710-ac7e-409d-9d68-671c5301c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.344433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad710-ac7e-409d-9d68-671c5301c7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.344721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad710-ac7e-409d-9d68-671c5301c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.345747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad710-ac7e-409d-9d68-671c5301c7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.346591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ad710-ac7e-409d-9d68-671c5301c7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.348540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ad710-ac7e-409d-9d68-671c5301c7d7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.607835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 948/1894] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:26 2022] PUT /identity/v3/projects/26182f5095e94f26a90b9dc0d27af3ae/users/4a46b603542744eeb1aa5f75772bafd7/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:26.607835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.657278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b064427-e0bb-489b-b6b7-1103c3eddcd7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.675579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.675579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.675579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.675579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] PATH_INFO: `/v3/projects/12c5abe6a77e42428011fcdca169364a/users/6769f9c2b0c44920ae9132984c0df2f2/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.680585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c5abe6a77e42428011fcdca169364a, user_id=6769f9c2b0c44920ae9132984c0df2f2, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.683459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.781379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 947/1895] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:26 2022] POST /identity/v3/users => generated 303 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:26.783414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6fd21-c8b6-44fa-9fb6-f37772d88dcc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.783709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6fd21-c8b6-44fa-9fb6-f37772d88dcc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.783993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6fd21-c8b6-44fa-9fb6-f37772d88dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.815299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9cc6fd21-c8b6-44fa-9fb6-f37772d88dcc None None] MFA Rules not processed for user `4a46b603542744eeb1aa5f75772bafd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:26.816565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9cc6fd21-c8b6-44fa-9fb6-f37772d88dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:26.889746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 948/1896] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:26.890528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6fd21-c8b6-44fa-9fb6-f37772d88dcc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.901982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6fd21-c8b6-44fa-9fb6-f37772d88dcc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.906911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.908127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.909160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.909445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.910209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.911854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.928844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 949/1897] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:26.928844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:26.941058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8eb47-53bc-4094-80ad-813ae3bcb452 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:26.946473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772585a7-e035-4175-bc8c-0907040bd747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:26.947624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772585a7-e035-4175-bc8c-0907040bd747 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:26.947624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772585a7-e035-4175-bc8c-0907040bd747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:26.947945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772585a7-e035-4175-bc8c-0907040bd747 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:26.948565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772585a7-e035-4175-bc8c-0907040bd747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:26.950788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772585a7-e035-4175-bc8c-0907040bd747 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:26.952616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 949/1898] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:26 2022] PUT /identity/v3/projects/12c5abe6a77e42428011fcdca169364a/users/6769f9c2b0c44920ae9132984c0df2f2/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:26.953832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.031844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835afd59-c827-4cc1-b701-68651cb4037d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.038886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.039789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.040074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.040350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] PATH_INFO: `/v3/projects/9a9ced0a3a764b81bdbe45b9c21e2299/users/b5a668f1438f424297244ae3e4392edd/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.057371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9ced0a3a764b81bdbe45b9c21e2299, user_id=b5a668f1438f424297244ae3e4392edd, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.060083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.102979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 950/1899] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:26 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:27.104353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182aaf-2933-4225-a00e-69c6b62df79c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.104653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182aaf-2933-4225-a00e-69c6b62df79c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.104998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182aaf-2933-4225-a00e-69c6b62df79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.133928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-28182aaf-2933-4225-a00e-69c6b62df79c None None] MFA Rules not processed for user `6769f9c2b0c44920ae9132984c0df2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:27.135155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28182aaf-2933-4225-a00e-69c6b62df79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:27.154217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 950/1900] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:26 2022] PUT /identity/v3/projects/9a9ced0a3a764b81bdbe45b9c21e2299/users/b5a668f1438f424297244ae3e4392edd/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:27.155552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.211713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 951/1901] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:27.216762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182aaf-2933-4225-a00e-69c6b62df79c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.228606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cb281-e5d8-48fd-9bc6-cc8e246bb543 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.234719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.235609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.240597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.240597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.240597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.240597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.315237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182aaf-2933-4225-a00e-69c6b62df79c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.321201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.325021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.325021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.325021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.325021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.325021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.336672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 952/1902] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:27.337623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.339324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:27.352303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 951/1903] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:27 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:27.353828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.364623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a33ddd-2d52-4b74-b535-df6960a62196 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.369737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.371056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.371343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.371627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] PATH_INFO: `/v3/projects/9a9ced0a3a764b81bdbe45b9c21e2299/users/b5a668f1438f424297244ae3e4392edd/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.374058 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9ced0a3a764b81bdbe45b9c21e2299, user_id=b5a668f1438f424297244ae3e4392edd, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.377540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.400995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 952/1904] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:27 2022] PUT /identity/v3/projects/9a9ced0a3a764b81bdbe45b9c21e2299/users/b5a668f1438f424297244ae3e4392edd/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:27.400995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.432056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b90868-1ed8-41eb-a185-eeeb928aceff admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.490546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f619a3d-d434-41f5-922c-b2d4e988ba92 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.505023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccd9f1-b5d0-49af-bccd-3b17829debb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.505824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd9f1-b5d0-49af-bccd-3b17829debb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.506152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd9f1-b5d0-49af-bccd-3b17829debb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.506433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd9f1-b5d0-49af-bccd-3b17829debb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.507216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccd9f1-b5d0-49af-bccd-3b17829debb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.508322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.509048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccd9f1-b5d0-49af-bccd-3b17829debb7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.509170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.509481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.509741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.524604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 953/1905] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:27.524604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.524604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:27.524604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.525203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.559955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] MFA Rules not processed for user `b5a668f1438f424297244ae3e4392edd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:27.561348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:27.609387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.612669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.624462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 953/1906] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:27.628188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.653937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b89aa-857e-4eeb-8d03-c8da56fe5f45 service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.659283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 954/1907] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:27.662145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.662870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.663154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.663428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.664186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.665896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.673239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.676367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:27.678890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 954/1908] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:27.683842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.694888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec27915-cc6e-4e3d-9a2e-54926c1162c1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.700375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d255a02-23ec-4a5d-a5ac-c921eb821a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.701435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d255a02-23ec-4a5d-a5ac-c921eb821a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.701722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d255a02-23ec-4a5d-a5ac-c921eb821a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.702004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d255a02-23ec-4a5d-a5ac-c921eb821a43 admin admin] PATH_INFO: `/v3/projects/60f9e3381ce9494b8c7d77b43d592f89/users/1ee504617d9d4287ae2645fb0b904303/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.722381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d255a02-23ec-4a5d-a5ac-c921eb821a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f9e3381ce9494b8c7d77b43d592f89, user_id=1ee504617d9d4287ae2645fb0b904303, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.725410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d255a02-23ec-4a5d-a5ac-c921eb821a43 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.758141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dda035-5473-438d-9576-f60dbc23b4cd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.762246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.771305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.771305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.771305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.776368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.778361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.791131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 955/1909] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:27.845457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.850933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 955/1910] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:27 2022] PUT /identity/v3/projects/60f9e3381ce9494b8c7d77b43d592f89/users/1ee504617d9d4287ae2645fb0b904303/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:27.863995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308584d-ba6f-4e51-a89a-7ba74f1cea48 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.872083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd433643-fd7a-4b0b-bafb-5075c6c23a59 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.872606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd433643-fd7a-4b0b-bafb-5075c6c23a59 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.873138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd433643-fd7a-4b0b-bafb-5075c6c23a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.883976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:27.915179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fd433643-fd7a-4b0b-bafb-5075c6c23a59 None None] MFA Rules not processed for user `1ee504617d9d4287ae2645fb0b904303`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:27.917646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd433643-fd7a-4b0b-bafb-5075c6c23a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:27.929711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.930383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.930664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.930939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:28.006580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:28.009726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:28.021418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 956/1911] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:28.026978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 956/1912] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:28.033197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:28.111375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad9127-9749-4219-a808-173c3bfdb47f service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:28.123409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:28.124229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:28.124447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:28.124739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:28.125513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:28.129166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:28.300834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 957/1913] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:28 2022] POST /identity/v3/projects => generated 375 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:28.302165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd433643-fd7a-4b0b-bafb-5075c6c23a59 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:28.318925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd433643-fd7a-4b0b-bafb-5075c6c23a59 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:28.422191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:28.427836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:28.427836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:28.427836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:28.427836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:28.427836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:28.667574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 957/1914] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:28 2022] POST /identity/v3/users => generated 294 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:28.671802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:28.684155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cb6c3-6a55-46a3-8e01-57c626a6e71d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:28.689708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:28.690502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:28.690815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:28.691092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:28.691846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:28.693591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:28.705974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 958/1915] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:28.710577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:28.724284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4cbfa-2eb7-4c28-8593-a1704e1aa0b5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:28.730522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:28.731778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:28.732088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:28.732369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] PATH_INFO: `/v3/projects/47f5dcf30e254a7fa4aa32f0652ef662/users/4c4f1553924b4725b2552f10905af55c/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:28.758507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f5dcf30e254a7fa4aa32f0652ef662, user_id=4c4f1553924b4725b2552f10905af55c, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:28.761714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:28.885253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 958/1916] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:28 2022] PUT /identity/v3/projects/47f5dcf30e254a7fa4aa32f0652ef662/users/4c4f1553924b4725b2552f10905af55c/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:28.890255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:28.905580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5695-362d-41bd-a1aa-2410a395f900 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:28.974316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623e9a6-0257-46f8-a1a1-ee59efab5856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:28.975012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e9a6-0257-46f8-a1a1-ee59efab5856 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:28.975297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e9a6-0257-46f8-a1a1-ee59efab5856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:28.975575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e9a6-0257-46f8-a1a1-ee59efab5856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:28.976322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623e9a6-0257-46f8-a1a1-ee59efab5856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:28.978020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623e9a6-0257-46f8-a1a1-ee59efab5856 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:28.989656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 959/1917] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:28.994047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:29.007930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f0769-e781-4d74-a75e-b58b2dbdc362 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:29.021416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:29.021416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:29.021416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:29.021416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] PATH_INFO: `/v3/projects/47f5dcf30e254a7fa4aa32f0652ef662/users/4c4f1553924b4725b2552f10905af55c/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:29.023721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f5dcf30e254a7fa4aa32f0652ef662, user_id=4c4f1553924b4725b2552f10905af55c, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:29.028310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:29.049936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 959/1918] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:28 2022] PUT /identity/v3/projects/47f5dcf30e254a7fa4aa32f0652ef662/users/4c4f1553924b4725b2552f10905af55c/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:29.061812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9171d-3ec5-4f84-8639-dc7c7c2bbd30 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:29.062130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9171d-3ec5-4f84-8639-dc7c7c2bbd30 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:29.062441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9171d-3ec5-4f84-8639-dc7c7c2bbd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:29.114552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0ae9171d-3ec5-4f84-8639-dc7c7c2bbd30 None None] MFA Rules not processed for user `4c4f1553924b4725b2552f10905af55c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:29.118547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ae9171d-3ec5-4f84-8639-dc7c7c2bbd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:29.250932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 960/1919] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:29.265409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:29.358834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08032c77-2f22-41b7-85a6-68cff9feb5aa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:29.364359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:29.365436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:29.365854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:29.366242 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:29.446584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:29.448183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:29.458978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 960/1920] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:29.766353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:30.160133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:30.818360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:31.760758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9171d-3ec5-4f84-8639-dc7c7c2bbd30 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:31.780261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9171d-3ec5-4f84-8639-dc7c7c2bbd30 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:31.787838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:31.789763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:31.793925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:31.793925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:31.927699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:31.930880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:31.948603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 961/1921] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:32.083622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:33.284577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:33.717945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:34.005683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:34.123681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4d75-94e8-4015-85c0-b185bb7da12c service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:34.129374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:34.130383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:34.130721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:34.131088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] PATH_INFO: `/v3/users/a41c687c751e425c93346421566687bd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:34.132570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41c687c751e425c93346421566687bd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:34.134732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:34.374208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:34.386370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeedc41-0dc4-487a-bc59-6bd20bca72c9 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:34.401038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a10962-f1c5-4b68-851d-4ea860b8e2c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:34.402468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a10962-f1c5-4b68-851d-4ea860b8e2c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:34.402973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a10962-f1c5-4b68-851d-4ea860b8e2c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:34.403455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a10962-f1c5-4b68-851d-4ea860b8e2c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:34.419670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a10962-f1c5-4b68-851d-4ea860b8e2c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:34.421223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a10962-f1c5-4b68-851d-4ea860b8e2c5 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:34.443076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 962/1922] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:34.475371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41c687c751e425c93346421566687bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:34.484837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41c687c751e425c93346421566687bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:34.501735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41c687c751e425c93346421566687bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:34.672743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41c687c751e425c93346421566687bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:34.676285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41c687c751e425c93346421566687bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:34.693239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 961/1923] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:34 2022] DELETE /identity/v3/users/a41c687c751e425c93346421566687bd => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:34.699496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:34.699974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:34.700375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:34.727537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:34.729477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:34.803186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 963/1924] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:34.821767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:34.821767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:34.922202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0558d4-f887-4d75-89c1-242cd5ca713b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:34.929058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:34.930255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:34.930775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:34.931247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:34.947816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:34.950818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:34.973916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 962/1925] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:35.330242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:35.577096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:35.578731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:35.587802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:35.594629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756be5b-37f8-4ea1-8193-6ac5d7ae1760 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:35.602577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:35.603690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:35.604002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:35.604279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] PATH_INFO: `/v3/projects/eee60a7fa3854c46bbf7e85a4aa49cab` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:35.606385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee60a7fa3854c46bbf7e85a4aa49cab)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:35.616922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:35.770784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee60a7fa3854c46bbf7e85a4aa49cab', 'request_id': 'req-79426d93-a600-4926-a065-2ee531117a3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:35.929903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee60a7fa3854c46bbf7e85a4aa49cab', 'request_id': 'req-79426d93-a600-4926-a065-2ee531117a3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:35.945574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee60a7fa3854c46bbf7e85a4aa49cab', 'request_id': 'req-79426d93-a600-4926-a065-2ee531117a3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:35.970715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] The token cache is being invalidate because project eee60a7fa3854c46bbf7e85a4aa49cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:35.972048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:35.974247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:35.978437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 964/1926] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:35 2022] DELETE /identity/v3/projects/eee60a7fa3854c46bbf7e85a4aa49cab => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:35.984611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:36.011170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f14204-7272-4aee-9a19-8f89f354d69f service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:36.170230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:36.172295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:36.174643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:36.175437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] PATH_INFO: `/v3/users/b5a668f1438f424297244ae3e4392edd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:36.178262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5a668f1438f424297244ae3e4392edd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:36.182513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:36.468158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5a668f1438f424297244ae3e4392edd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:36.478332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5a668f1438f424297244ae3e4392edd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:36.494254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5a668f1438f424297244ae3e4392edd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:36.608862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5a668f1438f424297244ae3e4392edd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:36.609658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-433bc9cf-86ca-4053-bc77-7600bcdc3cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5a668f1438f424297244ae3e4392edd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:36.618267 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 963/1927] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:35 2022] DELETE /identity/v3/users/b5a668f1438f424297244ae3e4392edd => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:37.302693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:37.317837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:37.319869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:37.352038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79426d93-a600-4926-a065-2ee531117a3c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:37.447045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:37.449281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:37.450026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:37.450580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] PATH_INFO: `/v3/projects/9a9ced0a3a764b81bdbe45b9c21e2299` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:37.452085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9ced0a3a764b81bdbe45b9c21e2299)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:37.456003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:37.637829 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:37.676347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9ced0a3a764b81bdbe45b9c21e2299', 'request_id': 'req-d7708699-f50f-4254-a513-9c9445ea9179'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:37.791639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9ced0a3a764b81bdbe45b9c21e2299', 'request_id': 'req-d7708699-f50f-4254-a513-9c9445ea9179'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:37.806075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9ced0a3a764b81bdbe45b9c21e2299', 'request_id': 'req-d7708699-f50f-4254-a513-9c9445ea9179'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:37.851580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] The token cache is being invalidate because project 9a9ced0a3a764b81bdbe45b9c21e2299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:37.852114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:37.857339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7708699-f50f-4254-a513-9c9445ea9179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:37.866245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 965/1928] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:37 2022] DELETE /identity/v3/projects/9a9ced0a3a764b81bdbe45b9c21e2299 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:37.875807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0650d-5807-4309-81f5-56962c951497 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:37.876151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0650d-5807-4309-81f5-56962c951497 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:37.876518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0650d-5807-4309-81f5-56962c951497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:37.908390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a2e0650d-5807-4309-81f5-56962c951497 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:37.909926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a2e0650d-5807-4309-81f5-56962c951497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:38.005263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58539-5092-4b6c-819e-907fd87dd261 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.010014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58539-5092-4b6c-819e-907fd87dd261 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.010014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58539-5092-4b6c-819e-907fd87dd261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:38.010394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 964/1929] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:38.050075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-eeb58539-5092-4b6c-819e-907fd87dd261 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:38.050075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eeb58539-5092-4b6c-819e-907fd87dd261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:38.076759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 966/1930] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:38.077754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0650d-5807-4309-81f5-56962c951497 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:38.092761 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0650d-5807-4309-81f5-56962c951497 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:38.099849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:38.100818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.101387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.101875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:38.103030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:38.103689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:38.105543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:38.128332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 965/1931] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:48:38.140544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb58539-5092-4b6c-819e-907fd87dd261 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:38.161905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb58539-5092-4b6c-819e-907fd87dd261 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:38.167153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:38.167973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.168506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.169007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:38.170092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:38.173815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:38.324447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 967/1932] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:38 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:38.330409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:38.352986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874763c5-c36d-4f27-955f-08734d8db354 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:38.526618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:38.535477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.535477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.535477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:38.535477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:38.537681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:38.688854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:38.701878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:38.709613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 966/1933] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:38 2022] POST /identity/v3/users => generated 293 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:38.713580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1056ad-8883-413a-8e7d-2f85c2149a90 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:38.718900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:38.719769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.720073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.720808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:38.722068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:38.724351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:38.738539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 968/1934] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:38.742864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:38.755699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe476d94-bb6c-41af-a76e-77122ebca321 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:38.761083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9402c-3695-4e8b-aa83-3e4ab33ccc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:38.761858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9402c-3695-4e8b-aa83-3e4ab33ccc62 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.762155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9402c-3695-4e8b-aa83-3e4ab33ccc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.762845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9402c-3695-4e8b-aa83-3e4ab33ccc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:38.763278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9402c-3695-4e8b-aa83-3e4ab33ccc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:38.765034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9402c-3695-4e8b-aa83-3e4ab33ccc62 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:38.787420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 967/1935] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:38.791620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:38.811712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d795c98-a323-4d5e-914d-6417ada37187 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:38.822963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:38.824097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.824428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.824708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] PATH_INFO: `/v3/projects/898ea8346c06459d950fcdae2da16948/users/0b09eea4ab67423d87e1cd62555b6fc5/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:38.853582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898ea8346c06459d950fcdae2da16948, user_id=0b09eea4ab67423d87e1cd62555b6fc5, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:38.859246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:38.973493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 969/1936] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:38 2022] PUT /identity/v3/projects/898ea8346c06459d950fcdae2da16948/users/0b09eea4ab67423d87e1cd62555b6fc5/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:38.991016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da2a5-3a1e-4f67-bbc3-039c584ef414 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:38.991698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da2a5-3a1e-4f67-bbc3-039c584ef414 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:38.992250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da2a5-3a1e-4f67-bbc3-039c584ef414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:39.026770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8c8da2a5-3a1e-4f67-bbc3-039c584ef414 None None] MFA Rules not processed for user `0b09eea4ab67423d87e1cd62555b6fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:39.028089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8c8da2a5-3a1e-4f67-bbc3-039c584ef414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:39.139312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 968/1937] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:39.144293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:39.285354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281126a4-c731-405f-8b0f-2ef897376a69 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:39.294559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:39.295261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:39.295541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:39.295861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:39.296683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:39.300128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:39.461221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 970/1938] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:39 2022] POST /identity/v3/projects => generated 387 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:39.462535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8da2a5-3a1e-4f67-bbc3-039c584ef414 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:39.480269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8da2a5-3a1e-4f67-bbc3-039c584ef414 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:39.579717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:39.579717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:39.579717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:39.579717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:39.580020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:39.586289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:39.843782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 969/1939] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:39 2022] POST /identity/v3/users => generated 300 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:39.848671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:39.859799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0894daf-7a09-4111-91c1-c943838fdadf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:39.865498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:39.866191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:39.866484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:39.866764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:39.867526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:39.869282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:39.882799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 971/1940] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:39.887605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:39.898615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d0464-1a39-443a-8510-b77b940ef1e0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:39.903387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:39.904316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:39.904599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:39.904985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] PATH_INFO: `/v3/projects/02ac292227a14118962a59a4a08d1716/users/f5e8939009ec49489ed02bb60171446d/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:39.931255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:39.934173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:39.934449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ac292227a14118962a59a4a08d1716, user_id=f5e8939009ec49489ed02bb60171446d, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:39.938255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:39.957090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983e56-4f3d-48a0-a011-6ca7288ce0c4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:39.964617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:39.965945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:39.966227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:39.966606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] PATH_INFO: `/v3/users/ce15ddb712fa42dc8d487fe22d0959bd` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:39.967938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce15ddb712fa42dc8d487fe22d0959bd)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:39.970772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:40.113212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:40.157265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 970/1941] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:39 2022] PUT /identity/v3/projects/02ac292227a14118962a59a4a08d1716/users/f5e8939009ec49489ed02bb60171446d/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:40.162573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:40.233419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce15ddb712fa42dc8d487fe22d0959bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.247092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae21232-0518-4893-b12a-5f37bc892111 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:40.255348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce15ddb712fa42dc8d487fe22d0959bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.258570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:40.258570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:40.258570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:40.259065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:40.259821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:40.261747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:40.275087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 971/1942] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:40.282854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:40.293999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65353de-fba1-4351-bf5f-527920dae5e2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:40.300146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:40.301103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:40.301408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:40.301654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] PATH_INFO: `/v3/projects/02ac292227a14118962a59a4a08d1716/users/f5e8939009ec49489ed02bb60171446d/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:40.303808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ac292227a14118962a59a4a08d1716, user_id=f5e8939009ec49489ed02bb60171446d, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:40.306550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:40.321534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 972/1943] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:40 2022] PUT /identity/v3/projects/02ac292227a14118962a59a4a08d1716/users/f5e8939009ec49489ed02bb60171446d/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:40.321534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:40.321534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:40.349726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce15ddb712fa42dc8d487fe22d0959bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.350499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce15ddb712fa42dc8d487fe22d0959bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.351351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f238371c-9041-4735-a16a-6bd043294956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce15ddb712fa42dc8d487fe22d0959bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.371685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 972/1944] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:39 2022] DELETE /identity/v3/users/ce15ddb712fa42dc8d487fe22d0959bd => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:40.373354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:40.373751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:40.374107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:40.416761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] MFA Rules not processed for user `f5e8939009ec49489ed02bb60171446d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:40.419849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:40.441089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900be1b-63fe-469d-a434-badfe7b83908 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:40.516923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:40.524433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 973/1945] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:40.540810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:40.542446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:40.552663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:40.553760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:40.556844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:40.556844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] PATH_INFO: `/v3/users/4a46b603542744eeb1aa5f75772bafd7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:40.557667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a46b603542744eeb1aa5f75772bafd7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:40.560130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:40.620949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b6d3-8e6c-44d1-985d-177c3c967b64 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:40.624820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:40.625435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:40.625745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:40.626550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:40.637035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:40.638524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:40.650861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 974/1946] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:40.748241 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:40.749773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:40.767426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae20c38-e7e3-46b5-a0ff-2bddf5c7dda1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:40.772790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:40.772790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:40.772790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:40.772790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] PATH_INFO: `/v3/projects/6d288974aac54fa681cfafa262d019eb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:40.778245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d288974aac54fa681cfafa262d019eb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:40.778245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:40.877323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a46b603542744eeb1aa5f75772bafd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.886731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a46b603542744eeb1aa5f75772bafd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.901871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a46b603542744eeb1aa5f75772bafd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:40.905050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d288974aac54fa681cfafa262d019eb', 'request_id': 'req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:41.037060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a46b603542744eeb1aa5f75772bafd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:41.037728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-88f6c33a-9493-4762-8c86-9c88473b5bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a46b603542744eeb1aa5f75772bafd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:41.045396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 973/1947] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:40 2022] DELETE /identity/v3/users/4a46b603542744eeb1aa5f75772bafd7 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:41.052609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.052941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.053227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.071985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:41.073309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:41.142999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 974/1948] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:41.151394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d288974aac54fa681cfafa262d019eb', 'request_id': 'req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:41.153289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.158577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:41.162105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d288974aac54fa681cfafa262d019eb', 'request_id': 'req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:41.181624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] The token cache is being invalidate because project 6d288974aac54fa681cfafa262d019eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:41.182001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:41.182702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08c6ffb5-1bf6-42c1-9d6a-a42b76704665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:41.184736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 975/1949] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:40 2022] DELETE /identity/v3/projects/6d288974aac54fa681cfafa262d019eb => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:41.189616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1b0e2-e7e1-4c5c-a1d6-3a43cd4590a6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.189915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1b0e2-e7e1-4c5c-a1d6-3a43cd4590a6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.190198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1b0e2-e7e1-4c5c-a1d6-3a43cd4590a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.213731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f6b1b0e2-e7e1-4c5c-a1d6-3a43cd4590a6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:41.214983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f6b1b0e2-e7e1-4c5c-a1d6-3a43cd4590a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:41.226038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 976/1950] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:41.235106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ecf9c-4444-4c98-aee5-9709f3ade34c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.235106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ecf9c-4444-4c98-aee5-9709f3ade34c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.236165 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ecf9c-4444-4c98-aee5-9709f3ade34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.258811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2fe65-c51b-492c-ac5b-4f3a454644be None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.259607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-859ecf9c-4444-4c98-aee5-9709f3ade34c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:41.261939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-859ecf9c-4444-4c98-aee5-9709f3ade34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:41.263915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.264991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.265593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.266052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.269265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:41.277180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 977/1951] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:41.281579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ecf9c-4444-4c98-aee5-9709f3ade34c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.284447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.285954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:41.291827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ecf9c-4444-4c98-aee5-9709f3ade34c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.296688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.297325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.297608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.298524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 975/1952] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:41.299009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.299581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:41.300026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.301796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:41.318807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 978/1953] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:48:41.321232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.337709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1d222-b4c2-40d5-849f-2fd2f379db16 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.346739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.351716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.351716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.351716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.351716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.354104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:41.547817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 976/1954] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:41 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:41.550833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.561397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b933cf3-f434-4b45-a4c3-94e70a4aef43 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.669316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.669949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.670228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.670509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.671260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.673413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:41.825814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 979/1955] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:41 2022] POST /identity/v3/users => generated 287 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:41.828152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.840079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8fbc1-4976-42c4-8762-ccfcd5b4638b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.844933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.845641 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.845924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.846458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.847299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.848995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:41.862476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.865005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:41.867998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 977/1956] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:41.868969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.885844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a4b72-bd5c-4367-964d-e5b3a980de82 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.888071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c553b7-f01a-4d6e-9b48-d4fe157f6c6b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.891295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.892877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.893448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.893582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] PATH_INFO: `/v3/projects/26182f5095e94f26a90b9dc0d27af3ae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.895200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] RBAC: Authorizing `identity:delete_project(project_id=26182f5095e94f26a90b9dc0d27af3ae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.897324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:41.898747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.898922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.899217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.899458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.900270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.902099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:41.913846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 978/1957] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:41.925689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:41.932666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47339b60-ada3-4323-916b-f2c74ac45830 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:41.937933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99839c7-2b63-4e46-9220-abcdb7fd7ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:41.939109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99839c7-2b63-4e46-9220-abcdb7fd7ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:41.939379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99839c7-2b63-4e46-9220-abcdb7fd7ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:41.939659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99839c7-2b63-4e46-9220-abcdb7fd7ae3 admin admin] PATH_INFO: `/v3/projects/49b85a23aa434bdfa46ead0c5ce3c236/users/a70a046814104b18860758604253341b/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:41.957079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99839c7-2b63-4e46-9220-abcdb7fd7ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b85a23aa434bdfa46ead0c5ce3c236, user_id=a70a046814104b18860758604253341b, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:41.962842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99839c7-2b63-4e46-9220-abcdb7fd7ae3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:42.101474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:42.107308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 979/1958] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:41 2022] PUT /identity/v3/projects/49b85a23aa434bdfa46ead0c5ce3c236/users/a70a046814104b18860758604253341b/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:42.118833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00430a8e-987a-4bde-be02-944946500013 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:42.119257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00430a8e-987a-4bde-be02-944946500013 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:42.119615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00430a8e-987a-4bde-be02-944946500013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:42.120663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26182f5095e94f26a90b9dc0d27af3ae', 'request_id': 'req-e56c3063-d8c0-4b78-8500-290818f7e83a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.159171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-00430a8e-987a-4bde-be02-944946500013 None None] MFA Rules not processed for user `a70a046814104b18860758604253341b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:42.160471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-00430a8e-987a-4bde-be02-944946500013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:42.237006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 980/1959] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:42.238002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00430a8e-987a-4bde-be02-944946500013 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:42.241431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-00430a8e-987a-4bde-be02-944946500013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:42.275495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26182f5095e94f26a90b9dc0d27af3ae', 'request_id': 'req-e56c3063-d8c0-4b78-8500-290818f7e83a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.283417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26182f5095e94f26a90b9dc0d27af3ae', 'request_id': 'req-e56c3063-d8c0-4b78-8500-290818f7e83a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.301169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] The token cache is being invalidate because project 26182f5095e94f26a90b9dc0d27af3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:42.301466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.302194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.303814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 980/1960] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:41 2022] DELETE /identity/v3/projects/26182f5095e94f26a90b9dc0d27af3ae => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:42.306758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:42.306758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:42.317444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00430a8e-987a-4bde-be02-944946500013 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:42.320003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c3063-d8c0-4b78-8500-290818f7e83a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:42.322291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:42.323187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:42.323481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:42.323759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] PATH_INFO: `/v3/users/0b09eea4ab67423d87e1cd62555b6fc5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:42.325032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b09eea4ab67423d87e1cd62555b6fc5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:42.325210 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:42.325875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:42.326155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:42.326426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:42.327081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:42.327185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:42.329468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:42.507949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 981/1961] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:42 2022] POST /identity/v3/projects => generated 371 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:42.508883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:42.510537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:42.578021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512991f-16ef-4f82-afea-f64a23fd3b68 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:42.582885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:42.583822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:42.584110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:42.584384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] PATH_INFO: `/v3/users/1ee504617d9d4287ae2645fb0b904303` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:42.585621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee504617d9d4287ae2645fb0b904303)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:42.587551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:42.755322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b09eea4ab67423d87e1cd62555b6fc5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.762090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b09eea4ab67423d87e1cd62555b6fc5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.774531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:42.775433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b09eea4ab67423d87e1cd62555b6fc5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.821225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee504617d9d4287ae2645fb0b904303'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.836918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee504617d9d4287ae2645fb0b904303'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.886876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b09eea4ab67423d87e1cd62555b6fc5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.887558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b09eea4ab67423d87e1cd62555b6fc5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.895716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 981/1962] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:42 2022] DELETE /identity/v3/users/0b09eea4ab67423d87e1cd62555b6fc5 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:42.896518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:42.897872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:42.969673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9a42c-374c-499f-a726-a176d6866e97 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:42.974963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:42.975658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:42.975946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:42.976256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:42.977057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:42.979031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:42.984076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee504617d9d4287ae2645fb0b904303'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.984750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee504617d9d4287ae2645fb0b904303'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:42.986421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33322f98-c744-453d-a081-b3c48b5c079a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee504617d9d4287ae2645fb0b904303'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:43.002806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 982/1963] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:42 2022] DELETE /identity/v3/users/1ee504617d9d4287ae2645fb0b904303 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:43.005111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.005677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.006106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.028806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:43.030079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:43.104732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 983/1964] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:43.119521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.121236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:43.123487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 982/1965] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:42 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:43.128544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.131246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:43.144474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9242145-fa5c-4e6c-9039-c2a24916ea98 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:43.149784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:43.150484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.150753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.151032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.156598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:43.156598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:43.164956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 983/1966] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:43.169230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.181010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402d8c1-d763-4bbf-b537-8301b2397bf5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:43.186225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:43.187126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.187413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.187689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] PATH_INFO: `/v3/projects/a43c3dd0492c4e19a6c9a937dff3f4a8/users/1bcde9fbd76643688446e5f965910e62/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.193562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aba526-7d2e-4ea2-9f3f-8a24004820ba None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:43.198384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:43.199124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.199443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.199728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.205201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43c3dd0492c4e19a6c9a937dff3f4a8, user_id=1bcde9fbd76643688446e5f965910e62, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:43.208001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:43.210190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:43.211722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:43.222394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 984/1967] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:43.368022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 984/1968] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:43 2022] PUT /identity/v3/projects/a43c3dd0492c4e19a6c9a937dff3f4a8/users/1bcde9fbd76643688446e5f965910e62/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:43.373453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.387224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdef0d-b0a7-470f-8e47-8e977b6fc552 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:43.452358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03eaa72-4795-4174-b3e2-24aa69eb75b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:43.453219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03eaa72-4795-4174-b3e2-24aa69eb75b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.453510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03eaa72-4795-4174-b3e2-24aa69eb75b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.453798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03eaa72-4795-4174-b3e2-24aa69eb75b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.454556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03eaa72-4795-4174-b3e2-24aa69eb75b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:43.456244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03eaa72-4795-4174-b3e2-24aa69eb75b1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:43.472914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 985/1969] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:43.473777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.486722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7baf5-9ab2-4577-a3f6-7eb19f3a2a83 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:43.492756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:43.493837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.494129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.494972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] PATH_INFO: `/v3/projects/a43c3dd0492c4e19a6c9a937dff3f4a8/users/1bcde9fbd76643688446e5f965910e62/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.498550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43c3dd0492c4e19a6c9a937dff3f4a8, user_id=1bcde9fbd76643688446e5f965910e62, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:43.502046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:43.514275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 985/1970] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:43 2022] PUT /identity/v3/projects/a43c3dd0492c4e19a6c9a937dff3f4a8/users/1bcde9fbd76643688446e5f965910e62/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:43.524811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.525115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.525443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.554625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] MFA Rules not processed for user `1bcde9fbd76643688446e5f965910e62`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:43.555906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:43.615328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:43.646150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 986/1971] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:43.660633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.664035 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:43.685860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.687486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:43.761512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c188-f949-46df-b580-96f8d09ff41f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:43.767530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:43.768512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.768994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.769412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.778189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:43.792400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:43.794564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:43.805982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012bb93-bd11-4a78-bf7e-98f193d25151 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:43.819671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:43.820018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 986/1972] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:43.820410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:43.820849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:43.821136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] PATH_INFO: `/v3/projects/60f9e3381ce9494b8c7d77b43d592f89` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:43.822694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f9e3381ce9494b8c7d77b43d592f89)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:43.825179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:43.959471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:43.963939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:43.980353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f9e3381ce9494b8c7d77b43d592f89', 'request_id': 'req-b4911814-30af-4dc8-ba64-b05a5e1afdb7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:43.997197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3428-2796-44cc-8a50-37b2327f56f0 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:44.070067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:44.070942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:44.071226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:44.071589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] PATH_INFO: `/v3/projects/898ea8346c06459d950fcdae2da16948` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:44.072871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=898ea8346c06459d950fcdae2da16948)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:44.074802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:44.109688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f9e3381ce9494b8c7d77b43d592f89', 'request_id': 'req-b4911814-30af-4dc8-ba64-b05a5e1afdb7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.119102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f9e3381ce9494b8c7d77b43d592f89', 'request_id': 'req-b4911814-30af-4dc8-ba64-b05a5e1afdb7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.136921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] The token cache is being invalidate because project 60f9e3381ce9494b8c7d77b43d592f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:44.137204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.137943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.139559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 987/1973] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:43 2022] DELETE /identity/v3/projects/60f9e3381ce9494b8c7d77b43d592f89 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:44.145096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:44.146498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:44.160551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4911814-30af-4dc8-ba64-b05a5e1afdb7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:44.165684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:44.166621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:44.166905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:44.167214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] PATH_INFO: `/v3/users/4c4f1553924b4725b2552f10905af55c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:44.168466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c4f1553924b4725b2552f10905af55c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:44.170489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:44.205967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '898ea8346c06459d950fcdae2da16948', 'request_id': 'req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.212564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '898ea8346c06459d950fcdae2da16948', 'request_id': 'req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.292962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '898ea8346c06459d950fcdae2da16948', 'request_id': 'req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.309229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] The token cache is being invalidate because project 898ea8346c06459d950fcdae2da16948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:44.309522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.310194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.311828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 987/1974] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:43 2022] DELETE /identity/v3/projects/898ea8346c06459d950fcdae2da16948 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:44.316454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:44.317864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:44.386356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddf034-81a8-4e99-898a-ee79b8fde3f6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:44.392420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:44.393391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:44.393679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:44.393979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] PATH_INFO: `/v3/users/f5e8939009ec49489ed02bb60171446d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:44.395232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5e8939009ec49489ed02bb60171446d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:44.397412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:44.426295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c4f1553924b4725b2552f10905af55c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.443997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c4f1553924b4725b2552f10905af55c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.615036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c4f1553924b4725b2552f10905af55c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.615892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c4f1553924b4725b2552f10905af55c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.616678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c4f1553924b4725b2552f10905af55c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.631400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 988/1975] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:44 2022] DELETE /identity/v3/users/4c4f1553924b4725b2552f10905af55c => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:44.668159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5e8939009ec49489ed02bb60171446d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.677297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5e8939009ec49489ed02bb60171446d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.704095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5e8939009ec49489ed02bb60171446d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.869558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5e8939009ec49489ed02bb60171446d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.870375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5e8939009ec49489ed02bb60171446d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:44.878054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 988/1976] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:44 2022] DELETE /identity/v3/users/f5e8939009ec49489ed02bb60171446d => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:45.158574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:45.162642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.195803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:45.205831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.247763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775d295-b491-427f-8e57-c29eb927e2fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:45.305445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4643a2-d3dc-4ac2-a904-4c2fd19ee977 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:45.305445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:45.305445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.305445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.306006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] PATH_INFO: `/v3/projects/47f5dcf30e254a7fa4aa32f0652ef662` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.306006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f5dcf30e254a7fa4aa32f0652ef662)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:45.306616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:45.337814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:45.338767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.339052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.339331 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] PATH_INFO: `/v3/users/a70a046814104b18860758604253341b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.341157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70a046814104b18860758604253341b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:45.343443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:45.377495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:45.458784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f5dcf30e254a7fa4aa32f0652ef662', 'request_id': 'req-184bf0f3-69ad-4363-9c34-32667775487a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.466059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f5dcf30e254a7fa4aa32f0652ef662', 'request_id': 'req-184bf0f3-69ad-4363-9c34-32667775487a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.614533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:45.616123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f5dcf30e254a7fa4aa32f0652ef662', 'request_id': 'req-184bf0f3-69ad-4363-9c34-32667775487a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.627821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] The token cache is being invalidate because project 47f5dcf30e254a7fa4aa32f0652ef662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:45.628124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.628798 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.629383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70a046814104b18860758604253341b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.630371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 989/1977] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:45 2022] DELETE /identity/v3/projects/47f5dcf30e254a7fa4aa32f0652ef662 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:45.633676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:45.633676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.643588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70a046814104b18860758604253341b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.699944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bf0f3-69ad-4363-9c34-32667775487a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:45.706078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:45.706961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.707248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.707575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] PATH_INFO: `/v3/projects/02ac292227a14118962a59a4a08d1716` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.709045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ac292227a14118962a59a4a08d1716)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:45.710958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:45.757027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70a046814104b18860758604253341b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.757681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70a046814104b18860758604253341b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.758317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d138ec32-8dfb-468a-93e5-51bdbb8bab1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70a046814104b18860758604253341b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.773084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 989/1978] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:45 2022] DELETE /identity/v3/users/a70a046814104b18860758604253341b => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:45.777647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f8d93-a416-4828-86c5-8fee6ced9180 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.778382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f8d93-a416-4828-86c5-8fee6ced9180 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.778382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f8d93-a416-4828-86c5-8fee6ced9180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.796533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-138f8d93-a416-4828-86c5-8fee6ced9180 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:45.797997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-138f8d93-a416-4828-86c5-8fee6ced9180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.858448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ac292227a14118962a59a4a08d1716', 'request_id': 'req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.865940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ac292227a14118962a59a4a08d1716', 'request_id': 'req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.874615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 990/1979] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:45.876653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c5686-10dd-408a-9d52-45a821ea7cc2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.876958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c5686-10dd-408a-9d52-45a821ea7cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.877372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c5686-10dd-408a-9d52-45a821ea7cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.896399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4d2c5686-10dd-408a-9d52-45a821ea7cc2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:45.897695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d2c5686-10dd-408a-9d52-45a821ea7cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.972130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 991/1980] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:45.973469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c16d-e5e7-4667-a4ed-6a01fb968ce6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.973758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c16d-e5e7-4667-a4ed-6a01fb968ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.974040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c16d-e5e7-4667-a4ed-6a01fb968ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.985854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ac292227a14118962a59a4a08d1716', 'request_id': 'req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.995363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0d41c16d-e5e7-4667-a4ed-6a01fb968ce6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:45.996581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d41c16d-e5e7-4667-a4ed-6a01fb968ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.998639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] The token cache is being invalidate because project 02ac292227a14118962a59a4a08d1716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:45.998930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.999636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:46.001716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 990/1981] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:45 2022] DELETE /identity/v3/projects/02ac292227a14118962a59a4a08d1716 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:46.002606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.003969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.014343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 992/1982] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:46.014343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0c56-d553-46d9-8d1a-521e71b01627 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.014343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0c56-d553-46d9-8d1a-521e71b01627 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.014343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0c56-d553-46d9-8d1a-521e71b01627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.033760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4d0c0c56-d553-46d9-8d1a-521e71b01627 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:46.034981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d0c0c56-d553-46d9-8d1a-521e71b01627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.043109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 993/1983] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:46.043937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c0c56-d553-46d9-8d1a-521e71b01627 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.055180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c0c56-d553-46d9-8d1a-521e71b01627 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.062537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1175f-ee55-4e0f-a153-ed9ba7b84793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.063784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1175f-ee55-4e0f-a153-ed9ba7b84793 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.064273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1175f-ee55-4e0f-a153-ed9ba7b84793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.064761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1175f-ee55-4e0f-a153-ed9ba7b84793 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.065961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1175f-ee55-4e0f-a153-ed9ba7b84793 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:46.066431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1175f-ee55-4e0f-a153-ed9ba7b84793 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.073388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1175f-ee55-4e0f-a153-ed9ba7b84793 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.081501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 994/1984] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:46.083359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2a185-6bc9-4e2c-9764-639368082092 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.083653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2a185-6bc9-4e2c-9764-639368082092 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.083941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2a185-6bc9-4e2c-9764-639368082092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.088133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e55ae-fec5-4f98-bf3f-7e0862088cbd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.092524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.093456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.093780 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.094066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.097912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.114584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.117519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.120593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-49c2a185-6bc9-4e2c-9764-639368082092 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:46.121940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-49c2a185-6bc9-4e2c-9764-639368082092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.129212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 991/1985] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:46.130144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.134211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 995/1986] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:46.135381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2a185-6bc9-4e2c-9764-639368082092 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.142142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176f7f9-22c1-4b2e-96aa-1d7c6f1ef03a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.147993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.148855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.149381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.150014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.150132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2a185-6bc9-4e2c-9764-639368082092 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.150935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.154849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.160290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.162369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.162888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.163702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.165316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:46.166015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.168082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.192840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 996/1987] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:46.194325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.206092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb46a-15d5-4c41-a747-667e43d85cfe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.210789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.211484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.211768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.212049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.212748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.214760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.294605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 992/1988] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:46.298372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.340956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 997/1989] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:46.345511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.370038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17f4c4-e1d0-4af4-b45a-678a0471f4f9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.433787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7234c6-16fb-444d-99a1-121fe855d5ef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.433995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.434613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.434898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.435179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.436024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.437992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.442247 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.443161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.443604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.444017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.444882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.447401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.606207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 993/1990] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:46.609330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.619760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcafbe-451b-45b4-8b68-52fb35901b1b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.624529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.625232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.625547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.626553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.627186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.628878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.638292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 998/1991] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:46.639864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 994/1992] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:46.643266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.644925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.655017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30db3-2951-45da-b161-6d7c796b9fed admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.655197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597da2-db82-438c-9b69-a7d112f94ca0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.660010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.660326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.660684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.660996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.661088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.661298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.661392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.661552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.662017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.662316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.663709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.663895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.675094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 999/1993] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:46.680380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 995/1994] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:46.680536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.681431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.691252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94087cc-52de-4e59-b448-8f31d89b16ef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.697157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.697309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abfbe8-6ed6-4a48-9d3d-0a3a9345c65b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.697851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.698141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.698420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.699183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.700869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.710227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.711309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.711462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.711736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] PATH_INFO: `/v3/projects/e8ea247cf5a540768bbbc7260b74f11c/users/49e86a3f10b64ef38664013e197782c3/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.716240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1000/1995] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:46.717366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.719080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.730087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ea247cf5a540768bbbc7260b74f11c, user_id=49e86a3f10b64ef38664013e197782c3, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.732753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.733301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4ecad-a83b-4c6f-9876-a163a00ce33b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.739497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.740599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.740928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.741213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] PATH_INFO: `/v3/projects/49b85a23aa434bdfa46ead0c5ce3c236` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.742433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b85a23aa434bdfa46ead0c5ce3c236)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.744519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.813304 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 996/1996] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:46 2022] PUT /identity/v3/projects/e8ea247cf5a540768bbbc7260b74f11c/users/49e86a3f10b64ef38664013e197782c3/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:46.815810 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.865953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b85a23aa434bdfa46ead0c5ce3c236', 'request_id': 'req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:46.884750 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c580f5a-5454-402b-8fc4-dfa87b677c16 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.965427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.966355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.966645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.966928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] PATH_INFO: `/v3/projects/65dc25c2563648ad886d17fb53c77bc3/users/d9e181ebd7cd41ed81a0ba51c46b3f34/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.983901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=65dc25c2563648ad886d17fb53c77bc3, user_id=d9e181ebd7cd41ed81a0ba51c46b3f34, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.987580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.004101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:47.004613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b85a23aa434bdfa46ead0c5ce3c236', 'request_id': 'req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.019261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b85a23aa434bdfa46ead0c5ce3c236', 'request_id': 'req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.035833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] The token cache is being invalidate because project 49b85a23aa434bdfa46ead0c5ce3c236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:47.036231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.037023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-743b9c83-98e4-4d98-8fd3-ba9dd9c20f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.038814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1001/1997] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:46 2022] DELETE /identity/v3/projects/49b85a23aa434bdfa46ead0c5ce3c236 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:47.041939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.041939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.041939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.059689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:47.060988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:47.073268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1002/1998] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:47.074253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:47.078971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:47.079581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 997/1999] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:46 2022] PUT /identity/v3/projects/65dc25c2563648ad886d17fb53c77bc3/users/d9e181ebd7cd41ed81a0ba51c46b3f34/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:47.086068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:47.093953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:47.102773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ea420-9b5e-4861-998d-14a2605f79a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.162199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b069ee-1b15-4d98-8008-7c7f8b5d4dad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.166198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd53d4-0226-4417-8fc6-f1b688da1d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.169774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd53d4-0226-4417-8fc6-f1b688da1d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.169774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd53d4-0226-4417-8fc6-f1b688da1d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.169774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd53d4-0226-4417-8fc6-f1b688da1d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.170045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.170538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.170996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.171273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] PATH_INFO: `/v3/users/1bcde9fbd76643688446e5f965910e62` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.172619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bcde9fbd76643688446e5f965910e62)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.175605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.179515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd53d4-0226-4417-8fc6-f1b688da1d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.181243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd53d4-0226-4417-8fc6-f1b688da1d9c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.191970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 998/2000] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:47.193346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.193643 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.193921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.224086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] MFA Rules not processed for user `d9e181ebd7cd41ed81a0ba51c46b3f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:47.225363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:47.246995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:47.293692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 999/2001] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:47.305488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:47.306861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:47.370275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60cd1-1cca-418a-b1af-4623265a4534 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.374585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.375242 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.375525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.376422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.385892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.387333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.398217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1000/2002] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:47.420925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:47.423481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:47.439204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bd700-fe15-4d47-86f5-fd2e52f97a0e service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.444105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.445092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.445401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.445679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] PATH_INFO: `/v3/users/d9e181ebd7cd41ed81a0ba51c46b3f34` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.446950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e181ebd7cd41ed81a0ba51c46b3f34)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.448947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.472427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bcde9fbd76643688446e5f965910e62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.487641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bcde9fbd76643688446e5f965910e62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.607074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bcde9fbd76643688446e5f965910e62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.607621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bcde9fbd76643688446e5f965910e62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.608257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c8b9132-ae93-4474-b291-77b63ab85af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bcde9fbd76643688446e5f965910e62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.623469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1003/2003] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:47 2022] DELETE /identity/v3/users/1bcde9fbd76643688446e5f965910e62 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:47.777256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e181ebd7cd41ed81a0ba51c46b3f34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.785515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e181ebd7cd41ed81a0ba51c46b3f34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.798749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e181ebd7cd41ed81a0ba51c46b3f34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.921017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e181ebd7cd41ed81a0ba51c46b3f34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.921680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e181ebd7cd41ed81a0ba51c46b3f34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:47.945452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1001/2004] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:47 2022] DELETE /identity/v3/users/d9e181ebd7cd41ed81a0ba51c46b3f34 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:47.948458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.948960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.949293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.970423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:47.972508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.072271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1004/2005] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:48.080218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.081797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.151187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73f018-eab2-4da2-b8bb-112b563b923c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.151187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bcabb-8ecd-403c-aada-454ef95443c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.151187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bcabb-8ecd-403c-aada-454ef95443c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.151187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bcabb-8ecd-403c-aada-454ef95443c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.151623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bcabb-8ecd-403c-aada-454ef95443c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.161765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bcabb-8ecd-403c-aada-454ef95443c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.163228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bcabb-8ecd-403c-aada-454ef95443c4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.174700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1002/2006] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:48.223992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.223992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.237681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa71b63-ea65-442d-862a-348da4142711 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.245968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.247033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.255164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.255164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] PATH_INFO: `/v3/projects/a43c3dd0492c4e19a6c9a937dff3f4a8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.255164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a43c3dd0492c4e19a6c9a937dff3f4a8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.255164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.415347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a43c3dd0492c4e19a6c9a937dff3f4a8', 'request_id': 'req-472b3401-4123-4a12-bd36-6d030401ce82'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:48.597831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a43c3dd0492c4e19a6c9a937dff3f4a8', 'request_id': 'req-472b3401-4123-4a12-bd36-6d030401ce82'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:48.609516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a43c3dd0492c4e19a6c9a937dff3f4a8', 'request_id': 'req-472b3401-4123-4a12-bd36-6d030401ce82'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:48.622365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] The token cache is being invalidate because project a43c3dd0492c4e19a6c9a937dff3f4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:48.622808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:48.623542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-472b3401-4123-4a12-bd36-6d030401ce82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:48.629373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1005/2007] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:48 2022] DELETE /identity/v3/projects/a43c3dd0492c4e19a6c9a937dff3f4a8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:48.631302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003acd-91c1-42ff-b0aa-dd5310a6a381 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.631998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003acd-91c1-42ff-b0aa-dd5310a6a381 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.632425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003acd-91c1-42ff-b0aa-dd5310a6a381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.663786 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ba003acd-91c1-42ff-b0aa-dd5310a6a381 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:48.665709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba003acd-91c1-42ff-b0aa-dd5310a6a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.766295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1003/2008] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:48.776559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.776856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.777193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.796552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:48.797951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.806405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1006/2009] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:48.810941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba003acd-91c1-42ff-b0aa-dd5310a6a381 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.822003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba003acd-91c1-42ff-b0aa-dd5310a6a381 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.826901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.827567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.827853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.828128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.829502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:48.829873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.831794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.844929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1004/2010] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:48:48.848481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.849246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.850295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.866398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60a19-0227-4d67-aea9-76c2e7bee843 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.871780 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff3b40-2dc7-4234-b57c-d003a7a5700b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.872674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.873676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.874961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.875258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.877150 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.878219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.879143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.879238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.879435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.879719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] PATH_INFO: `/v3/projects/65dc25c2563648ad886d17fb53c77bc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.881309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65dc25c2563648ad886d17fb53c77bc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.883591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.034826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1005/2011] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:48 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:49.036646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.053101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65dc25c2563648ad886d17fb53c77bc3', 'request_id': 'req-82528111-14d9-476e-b7e7-23f60728e604'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:49.101413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:49.124277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163ac10-028f-4194-9500-df47866be1d3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.157587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65dc25c2563648ad886d17fb53c77bc3', 'request_id': 'req-82528111-14d9-476e-b7e7-23f60728e604'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:49.167673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65dc25c2563648ad886d17fb53c77bc3', 'request_id': 'req-82528111-14d9-476e-b7e7-23f60728e604'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:49.182233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] The token cache is being invalidate because project 65dc25c2563648ad886d17fb53c77bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:49.182806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:49.184732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82528111-14d9-476e-b7e7-23f60728e604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:49.187787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1007/2012] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:48 2022] DELETE /identity/v3/projects/65dc25c2563648ad886d17fb53c77bc3 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:49.203655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.204082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.204461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.232366 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.233119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.233472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.233722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.234496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.237294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.270677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] MFA Rules not processed for user `49e86a3f10b64ef38664013e197782c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:49.272017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:49.363282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1006/2013] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:49 2022] POST /identity/v3/users => generated 284 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:49.365266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.367976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:49.402835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0acc4-340a-4a24-ad0b-a685396280d2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.428323 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1008/2014] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:49.434870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.436135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.436542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.436958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.437959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.438138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.439766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:49.440107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.451566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1007/2015] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:49.458211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.469174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710e037-3730-43e1-94e0-c95c365557fc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.476450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.477331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.477737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.478097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.479675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.480880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.501934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1008/2016] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:49.504449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.514995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274a933-6c3e-457f-b689-79f218d5a4f6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.520550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.521513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.521805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.522149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] PATH_INFO: `/v3/projects/b307fbb774794cdbb8bd588c122bfa30/users/d8bf63a5668e4a69b95ac1f1d21a75aa/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.522614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861836-0aff-47e7-bdbc-3e388a540105 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.527622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af5652-9657-49f6-afdd-58b17ee48190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.528289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5652-9657-49f6-afdd-58b17ee48190 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.528577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5652-9657-49f6-afdd-58b17ee48190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.528886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5652-9657-49f6-afdd-58b17ee48190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.546421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b307fbb774794cdbb8bd588c122bfa30, user_id=d8bf63a5668e4a69b95ac1f1d21a75aa, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.546524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af5652-9657-49f6-afdd-58b17ee48190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.546613 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.546695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af5652-9657-49f6-afdd-58b17ee48190 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.553952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1009/2017] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:49.661968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1009/2018] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:49 2022] PUT /identity/v3/projects/b307fbb774794cdbb8bd588c122bfa30/users/d8bf63a5668e4a69b95ac1f1d21a75aa/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:49.673632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3457c8-c9aa-4f2d-96a1-ede60aeaad30 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.674187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3457c8-c9aa-4f2d-96a1-ede60aeaad30 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.674507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3457c8-c9aa-4f2d-96a1-ede60aeaad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.678489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:49.706528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9f3457c8-c9aa-4f2d-96a1-ede60aeaad30 None None] MFA Rules not processed for user `d8bf63a5668e4a69b95ac1f1d21a75aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:49.707743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f3457c8-c9aa-4f2d-96a1-ede60aeaad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:49.795585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1010/2019] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:49.796005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.851858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081d77-0db6-483f-bfe9-f2b3b58a96c1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.856898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.857582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.858022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.858214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.858927 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.861000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.958998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1010/2020] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:49 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:49.963315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3457c8-c9aa-4f2d-96a1-ede60aeaad30 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.975281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3457c8-c9aa-4f2d-96a1-ede60aeaad30 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:50.031706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:50.032968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:50.033338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:50.033665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:50.034406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:50.036412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:50.186359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1011/2021] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:49 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:50.191910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:50.202990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91070409-94aa-4240-94c9-a8ab71cdb04a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:50.208333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:50.209055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:50.209398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:50.209684 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:50.210422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:50.212040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:50.227056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1011/2022] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:50.232087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:50.242747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033fe35-e691-4489-a389-55b5f6e45851 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:50.247682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:50.248579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:50.248913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:50.249219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] PATH_INFO: `/v3/projects/2c774f4bbbca479a88039be4eefb22a7/users/4c6a04fafc1840beb6a6826717e927f2/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:50.265587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c774f4bbbca479a88039be4eefb22a7, user_id=4c6a04fafc1840beb6a6826717e927f2, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:50.268909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:50.382979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1012/2023] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:50 2022] PUT /identity/v3/projects/2c774f4bbbca479a88039be4eefb22a7/users/4c6a04fafc1840beb6a6826717e927f2/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:50.391633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:50.398960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8febf2-74b4-4538-a5f3-20a8a121199e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:50.462475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b2961-530e-4246-a82b-f0533d0e3780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:50.463148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b2961-530e-4246-a82b-f0533d0e3780 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:50.463435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b2961-530e-4246-a82b-f0533d0e3780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:50.463717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b2961-530e-4246-a82b-f0533d0e3780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:50.464558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b2961-530e-4246-a82b-f0533d0e3780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:50.466193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b2961-530e-4246-a82b-f0533d0e3780 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:50.476998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1012/2024] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:50.481448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:50.492698 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a98693-d76d-4b9b-b2e5-dfbf62c4a6e8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:50.497580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:50.498469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:50.498750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:50.499021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] PATH_INFO: `/v3/projects/2c774f4bbbca479a88039be4eefb22a7/users/4c6a04fafc1840beb6a6826717e927f2/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:50.501693 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c774f4bbbca479a88039be4eefb22a7, user_id=4c6a04fafc1840beb6a6826717e927f2, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:50.504556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:50.516761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1013/2025] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:50 2022] PUT /identity/v3/projects/2c774f4bbbca479a88039be4eefb22a7/users/4c6a04fafc1840beb6a6826717e927f2/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:50.529347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0672-cbcd-4e59-939d-50dcbe6cc8d6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:50.529662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0672-cbcd-4e59-939d-50dcbe6cc8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:50.529963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0672-cbcd-4e59-939d-50dcbe6cc8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:50.560337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e6c0672-cbcd-4e59-939d-50dcbe6cc8d6 None None] MFA Rules not processed for user `4c6a04fafc1840beb6a6826717e927f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:50.565004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e6c0672-cbcd-4e59-939d-50dcbe6cc8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:50.655943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1013/2026] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:50.668531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:50.745969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce2b32-e58b-4af7-a9d6-3815ff80aa69 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:50.749937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:50.751318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:50.751563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:50.751740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:50.762153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:50.763610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:50.775844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1014/2027] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:51.572509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2ac48-ee5c-4ef2-b7c3-f39fe658fe2a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:51.573050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2ac48-ee5c-4ef2-b7c3-f39fe658fe2a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:51.573178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2ac48-ee5c-4ef2-b7c3-f39fe658fe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:51.597070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-76c2ac48-ee5c-4ef2-b7c3-f39fe658fe2a None None] MFA Rules not processed for user `49e86a3f10b64ef38664013e197782c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:51.598490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-76c2ac48-ee5c-4ef2-b7c3-f39fe658fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:51.673541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1014/2028] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:52.432791 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:52.786017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:54.500063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:54.928159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:55.001559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3ca67-3f5c-493b-b802-e4e5f1ae0f05 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:55.008834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:55.010170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:55.010694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:55.011076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] PATH_INFO: `/v3/users/d8bf63a5668e4a69b95ac1f1d21a75aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:55.012838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8bf63a5668e4a69b95ac1f1d21a75aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:55.015370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:55.219962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8bf63a5668e4a69b95ac1f1d21a75aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:55.233968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8bf63a5668e4a69b95ac1f1d21a75aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:55.322738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8bf63a5668e4a69b95ac1f1d21a75aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:55.323280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8bf63a5668e4a69b95ac1f1d21a75aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:55.323893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8bf63a5668e4a69b95ac1f1d21a75aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:55.338525 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1015/2029] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:54 2022] DELETE /identity/v3/users/d8bf63a5668e4a69b95ac1f1d21a75aa => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:55.343130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:55.343546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:55.343836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:55.363100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=5,keystone:SELECT=81,keystone:DELETE=14 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:55.369991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:55.371206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:55.437590 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1015/2030] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:55.448127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:55.449566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:55.510254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f59a2-f635-4f23-b2d8-a1f6b4210c8a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:55.514337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:55.515027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:55.515337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:55.515614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:55.531596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:55.534642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:55.543684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1016/2031] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:55.567453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:56.097330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:56.098935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:56.112323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b39e6d-9c4a-413e-825d-dc1083ff47f6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:56.117237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:56.118188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:56.118570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:56.118921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] PATH_INFO: `/v3/projects/b307fbb774794cdbb8bd588c122bfa30` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:56.120268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b307fbb774794cdbb8bd588c122bfa30)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:56.122384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:56.276067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b307fbb774794cdbb8bd588c122bfa30', 'request_id': 'req-2408a410-71d2-44ad-b509-d7789d6ce3f5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.283949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b307fbb774794cdbb8bd588c122bfa30', 'request_id': 'req-2408a410-71d2-44ad-b509-d7789d6ce3f5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.374139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b307fbb774794cdbb8bd588c122bfa30', 'request_id': 'req-2408a410-71d2-44ad-b509-d7789d6ce3f5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.386937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] The token cache is being invalidate because project b307fbb774794cdbb8bd588c122bfa30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:56.392820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.392820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.392820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1016/2032] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:56 2022] DELETE /identity/v3/projects/b307fbb774794cdbb8bd588c122bfa30 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:56.393104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:56.393861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:56.404066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492dcca-dcbd-4c93-9826-db5dd1d87094 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:56.464111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:56.465020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:56.465308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:56.465595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] PATH_INFO: `/v3/users/4c6a04fafc1840beb6a6826717e927f2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:56.466783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6a04fafc1840beb6a6826717e927f2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:56.468734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:56.487426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:56.762546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6a04fafc1840beb6a6826717e927f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.781344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6a04fafc1840beb6a6826717e927f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.944760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6a04fafc1840beb6a6826717e927f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.945492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6a04fafc1840beb6a6826717e927f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.946139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a4e25bb-72c2-4058-bb31-986bedee8080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6a04fafc1840beb6a6826717e927f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:56.959196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1017/2033] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:56 2022] DELETE /identity/v3/users/4c6a04fafc1840beb6a6826717e927f2 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:57.163401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:57.380522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:58.262885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:58.264226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:58.289249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408a410-71d2-44ad-b509-d7789d6ce3f5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:58.387894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:58.389439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:58.389971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:58.390491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] PATH_INFO: `/v3/projects/2c774f4bbbca479a88039be4eefb22a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:58.393018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c774f4bbbca479a88039be4eefb22a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:58.397042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:58.557809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c774f4bbbca479a88039be4eefb22a7', 'request_id': 'req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:58.564340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c774f4bbbca479a88039be4eefb22a7', 'request_id': 'req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:58.677082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c774f4bbbca479a88039be4eefb22a7', 'request_id': 'req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:58.691716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] The token cache is being invalidate because project 2c774f4bbbca479a88039be4eefb22a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:58.692146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:58.693015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6d2ed3a-8f8d-40e1-bf21-8c41cdb43785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:58.695160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1017/2034] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:58 2022] DELETE /identity/v3/projects/2c774f4bbbca479a88039be4eefb22a7 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:58.700266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e40f8d-d0c4-4467-8bdb-17bc0958d08a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:58.700877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e40f8d-d0c4-4467-8bdb-17bc0958d08a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:58.700877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e40f8d-d0c4-4467-8bdb-17bc0958d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:58.721853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-17e40f8d-d0c4-4467-8bdb-17bc0958d08a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:58.723148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-17e40f8d-d0c4-4467-8bdb-17bc0958d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:58.786208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1018/2035] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:58.793680 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863c27b-3502-4eff-a49e-c5625c0cadbe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:58.794112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863c27b-3502-4eff-a49e-c5625c0cadbe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:58.794461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863c27b-3502-4eff-a49e-c5625c0cadbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:58.821309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6863c27b-3502-4eff-a49e-c5625c0cadbe None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:58.822592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6863c27b-3502-4eff-a49e-c5625c0cadbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:58.831538 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1018/2036] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:58.836082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e40f8d-d0c4-4467-8bdb-17bc0958d08a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:58.847894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e40f8d-d0c4-4467-8bdb-17bc0958d08a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:58.853126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:58.853825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:58.854112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:58.854388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:58.855140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:58.855456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:58.857282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:58.871017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1019/2037] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:48:58.876858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863c27b-3502-4eff-a49e-c5625c0cadbe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:58.888666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863c27b-3502-4eff-a49e-c5625c0cadbe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:58.896446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:58.897266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:58.897618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:58.898137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:58.899030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:58.901193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:58.915037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:59.023053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1019/2038] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:58 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:59.027469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:59.038896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ae539-a1e7-45c4-8eeb-1c0fa2461312 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:59.099832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:59.100509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:59.100799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:59.101096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:59.102650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:59.104682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:59.286726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1020/2039] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:59 2022] POST /identity/v3/users => generated 284 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:59.288124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:59.302797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:59.303137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592947c-44f2-4ac5-be21-2a02f4f1032e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:59.309254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:59.309934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:59.310287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:59.310623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:59.311437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:59.314778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:59.326647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1020/2040] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:48:59.334334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:59.345089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d6614-8075-4bd9-a954-ceddfc994a70 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:59.350556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86ccfb-9042-4d35-a087-8b8332a91402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:59.351243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86ccfb-9042-4d35-a087-8b8332a91402 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:59.351517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86ccfb-9042-4d35-a087-8b8332a91402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:59.351791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86ccfb-9042-4d35-a087-8b8332a91402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:59.352550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86ccfb-9042-4d35-a087-8b8332a91402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:59.354227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86ccfb-9042-4d35-a087-8b8332a91402 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:59.365722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1021/2041] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:59.370336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:59.381493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5d04-5476-4030-bf20-78fee7e2e098 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:59.386254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:59.387180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:59.387461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:59.387762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] PATH_INFO: `/v3/projects/fa3a875da4224d3b8d2f61d85ecb7975/users/6bbf4e7245ed4f2ca0d9061dc17d6d50/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:59.404024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3a875da4224d3b8d2f61d85ecb7975, user_id=6bbf4e7245ed4f2ca0d9061dc17d6d50, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:59.407395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:59.512435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1021/2042] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:59 2022] PUT /identity/v3/projects/fa3a875da4224d3b8d2f61d85ecb7975/users/6bbf4e7245ed4f2ca0d9061dc17d6d50/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:48:59.524510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be81bc-9186-4b40-8b4a-a99beba92cae None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:59.524826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be81bc-9186-4b40-8b4a-a99beba92cae None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:59.525214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be81bc-9186-4b40-8b4a-a99beba92cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:59.558588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7be81bc-9186-4b40-8b4a-a99beba92cae None None] MFA Rules not processed for user `6bbf4e7245ed4f2ca0d9061dc17d6d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:59.560650 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7be81bc-9186-4b40-8b4a-a99beba92cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:59.661716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1022/2043] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:48:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:59.669239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:59.772250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2e910-2a58-44bb-bfdf-474a1372d74f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:59.779353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:59.780193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:59.780485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:59.780820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:59.781678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:59.784814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:59.983614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1022/2044] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:59 2022] POST /identity/v3/projects => generated 369 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:59.985091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be81bc-9186-4b40-8b4a-a99beba92cae None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:59.996756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be81bc-9186-4b40-8b4a-a99beba92cae None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:00.055629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:00.055904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:00.056049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:00.056246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:00.056983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:00.059032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:00.221284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1023/2045] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:59 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:00.225236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:00.236430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e77205-c2e3-4915-837d-fc1c6299256b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:00.242196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:00.242339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:00.242339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:00.242553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:00.242850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:00.243623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:00.245388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:00.257701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1023/2046] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:00.262230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:00.273317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c480-8dc6-4ce3-88e0-1e4d56d571a6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:00.278091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:00.278952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:00.279277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:00.279523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] PATH_INFO: `/v3/projects/b102678e111249acb5f9b12c2ffa01ad/users/2decbea700934103af4e00ee96e06f87/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:00.295888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b102678e111249acb5f9b12c2ffa01ad, user_id=2decbea700934103af4e00ee96e06f87, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:00.299469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:00.483960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1024/2047] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:00 2022] PUT /identity/v3/projects/b102678e111249acb5f9b12c2ffa01ad/users/2decbea700934103af4e00ee96e06f87/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:00.492207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:00.503794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d049ded-bae2-4393-8ba2-ec01717c3236 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:00.515759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:00.569316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21473aff-713b-4a31-8260-d2859d0bfd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:00.570013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473aff-713b-4a31-8260-d2859d0bfd20 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:00.570305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473aff-713b-4a31-8260-d2859d0bfd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:00.570591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473aff-713b-4a31-8260-d2859d0bfd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:00.571334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21473aff-713b-4a31-8260-d2859d0bfd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:00.573053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21473aff-713b-4a31-8260-d2859d0bfd20 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:00.585210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1024/2048] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:00.589577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:00.601695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7d1fb-2b8c-4c31-acaf-a2c5dcc16424 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:00.606491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:00.607444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:00.607768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:00.608169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] PATH_INFO: `/v3/projects/b102678e111249acb5f9b12c2ffa01ad/users/2decbea700934103af4e00ee96e06f87/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:00.611419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b102678e111249acb5f9b12c2ffa01ad, user_id=2decbea700934103af4e00ee96e06f87, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:00.614549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:00.628238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1025/2049] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:00 2022] PUT /identity/v3/projects/b102678e111249acb5f9b12c2ffa01ad/users/2decbea700934103af4e00ee96e06f87/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:00.643507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ec207-ead2-4c8a-a4e4-04c201b89e7d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:00.643803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ec207-ead2-4c8a-a4e4-04c201b89e7d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:00.644094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ec207-ead2-4c8a-a4e4-04c201b89e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:00.674191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cb1ec207-ead2-4c8a-a4e4-04c201b89e7d None None] MFA Rules not processed for user `2decbea700934103af4e00ee96e06f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:00.675463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb1ec207-ead2-4c8a-a4e4-04c201b89e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:00.758226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1025/2050] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:00 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:00.770257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:00.772311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:00.836451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32193f4-dce9-40ab-9af6-bf8940e2b484 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:00.840121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:00.840819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:00.841109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:00.841387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:00.851513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:00.853056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:00.863742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1026/2051] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:00 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:01.057893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ec207-ead2-4c8a-a4e4-04c201b89e7d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:01.069495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ec207-ead2-4c8a-a4e4-04c201b89e7d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:01.073739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7337d7e-0605-42c5-97b9-58c129ce0dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:01.074347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7337d7e-0605-42c5-97b9-58c129ce0dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:01.074656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7337d7e-0605-42c5-97b9-58c129ce0dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:01.074913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7337d7e-0605-42c5-97b9-58c129ce0dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:01.137621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7337d7e-0605-42c5-97b9-58c129ce0dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:01.139393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7337d7e-0605-42c5-97b9-58c129ce0dc7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:01.150079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1026/2052] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:01.458185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:01.795705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:03.082976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:03.635926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:03.882120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:03.882975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:03.955460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad5307-a135-4001-a0de-353adc435403 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:03.960517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:03.961319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:03.961606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:03.961884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] PATH_INFO: `/v3/users/6769f9c2b0c44920ae9132984c0df2f2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:03.963080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6769f9c2b0c44920ae9132984c0df2f2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:03.965276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:04.290641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6769f9c2b0c44920ae9132984c0df2f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.306691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6769f9c2b0c44920ae9132984c0df2f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.425969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6769f9c2b0c44920ae9132984c0df2f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.427044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6769f9c2b0c44920ae9132984c0df2f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.428039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6769f9c2b0c44920ae9132984c0df2f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.446060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1027/2053] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:03 2022] DELETE /identity/v3/users/6769f9c2b0c44920ae9132984c0df2f2 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:04.450581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:04.450876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:04.451155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:04.473545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:04.474764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:04.570357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1027/2054] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:04.580287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:04.581745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:04.652939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b46334-34f7-4c44-8e72-20910cf8da0f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:04.657440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b509d10-0a05-4ff6-bb2c-f645d2b94aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:04.658207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509d10-0a05-4ff6-bb2c-f645d2b94aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:04.658493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509d10-0a05-4ff6-bb2c-f645d2b94aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:04.658775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509d10-0a05-4ff6-bb2c-f645d2b94aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:04.669417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b509d10-0a05-4ff6-bb2c-f645d2b94aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:04.671725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b509d10-0a05-4ff6-bb2c-f645d2b94aa9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:04.690592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1028/2055] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:05.170678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:05.240982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:05.245375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:05.247228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:05.276263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11de32-0d41-4cd8-b465-2e29aef43992 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:05.284336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:05.285552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:05.285911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:05.286266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] PATH_INFO: `/v3/projects/12c5abe6a77e42428011fcdca169364a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:05.287948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c5abe6a77e42428011fcdca169364a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:05.290497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:05.483938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c5abe6a77e42428011fcdca169364a', 'request_id': 'req-a20aeb35-3289-4a88-88e4-b145c6e005c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.494041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c5abe6a77e42428011fcdca169364a', 'request_id': 'req-a20aeb35-3289-4a88-88e4-b145c6e005c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.626529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c5abe6a77e42428011fcdca169364a', 'request_id': 'req-a20aeb35-3289-4a88-88e4-b145c6e005c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.642018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] The token cache is being invalidate because project 12c5abe6a77e42428011fcdca169364a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:05.642249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.643052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a20aeb35-3289-4a88-88e4-b145c6e005c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.644652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1028/2056] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:05 2022] DELETE /identity/v3/projects/12c5abe6a77e42428011fcdca169364a => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:05.650276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aea707-16ac-428f-9904-5ece35eeea70 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:05.650800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aea707-16ac-428f-9904-5ece35eeea70 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:05.650891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aea707-16ac-428f-9904-5ece35eeea70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:05.672850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-85aea707-16ac-428f-9904-5ece35eeea70 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:05.674182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-85aea707-16ac-428f-9904-5ece35eeea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:05.757497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1029/2057] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:05.768263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d210-31ce-4fec-a297-42aa8b035d62 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:05.768928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d210-31ce-4fec-a297-42aa8b035d62 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:05.768928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d210-31ce-4fec-a297-42aa8b035d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:05.789742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-92a4d210-31ce-4fec-a297-42aa8b035d62 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:05.790995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92a4d210-31ce-4fec-a297-42aa8b035d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:05.800565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1029/2058] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:05.805079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aea707-16ac-428f-9904-5ece35eeea70 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:05.816460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aea707-16ac-428f-9904-5ece35eeea70 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:05.821637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:05.822362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:05.822648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:05.822928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:05.823701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:05.824040 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:05.825776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:05.845993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1030/2059] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:05.850623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4d210-31ce-4fec-a297-42aa8b035d62 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:05.863825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4d210-31ce-4fec-a297-42aa8b035d62 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:05.869094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:05.869817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:05.870103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:05.870381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:05.871082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:05.873036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.043571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1030/2060] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:05 2022] POST /identity/v3/projects => generated 397 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:06.051085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.065302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c41875-1823-4998-a4e6-430685de82f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.133294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.133989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.134269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.134541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.135249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.137280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.277713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1031/2061] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:06.286234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.308172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7c56-33ce-4501-a891-e0a47e5169bc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.317885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.319019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.319531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.320050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.321732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.325105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.348370 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1031/2062] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:49:06.352946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.365501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb815d4-1803-4e86-8f90-2971f307391b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.372968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ab446-bf17-43a8-b950-588ef9dd14c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.373668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ab446-bf17-43a8-b950-588ef9dd14c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.373956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ab446-bf17-43a8-b950-588ef9dd14c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.374235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ab446-bf17-43a8-b950-588ef9dd14c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.374991 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ab446-bf17-43a8-b950-588ef9dd14c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.376989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ab446-bf17-43a8-b950-588ef9dd14c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.409320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1032/2063] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:49:06.411444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.435203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33679-edf8-42e3-be0f-94c3f3b8753d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.443477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.445186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.445720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.446250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] PATH_INFO: `/v3/projects/455909c460574d7c9621fa9802970c48/users/169b8104259b47958ec674b98794bd26/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.479184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455909c460574d7c9621fa9802970c48, user_id=169b8104259b47958ec674b98794bd26, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.486651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.620820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1032/2064] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:06 2022] PUT /identity/v3/projects/455909c460574d7c9621fa9802970c48/users/169b8104259b47958ec674b98794bd26/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:06.634798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.635085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.635349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.665548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] MFA Rules not processed for user `169b8104259b47958ec674b98794bd26`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:06.666832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:06.742624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1033/2065] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:06.755479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.757596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:06.826214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f10300-d443-4c80-9bbf-a4ea9684e122 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.831183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bfc2-dd75-46eb-8d51-7d93521c8f55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.831864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bfc2-dd75-46eb-8d51-7d93521c8f55 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.832307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bfc2-dd75-46eb-8d51-7d93521c8f55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.832586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bfc2-dd75-46eb-8d51-7d93521c8f55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.841593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:06.845633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9bfc2-dd75-46eb-8d51-7d93521c8f55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.846439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9bfc2-dd75-46eb-8d51-7d93521c8f55 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.857271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1033/2066] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:07.230827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.232406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:07.325371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17028ce6-d608-4be9-855d-dc0d758c03a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:07.333162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:07.334616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.335088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.336097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] PATH_INFO: `/v3/users/6bbf4e7245ed4f2ca0d9061dc17d6d50` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.338273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbf4e7245ed4f2ca0d9061dc17d6d50)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:07.348442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.692063 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbf4e7245ed4f2ca0d9061dc17d6d50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:07.705249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbf4e7245ed4f2ca0d9061dc17d6d50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:07.854078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbf4e7245ed4f2ca0d9061dc17d6d50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:07.854747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbf4e7245ed4f2ca0d9061dc17d6d50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:07.855381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbf4e7245ed4f2ca0d9061dc17d6d50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:07.871138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1034/2067] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:07 2022] DELETE /identity/v3/users/6bbf4e7245ed4f2ca0d9061dc17d6d50 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:07.876478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.877186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.877186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.896328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:07.898232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:07.945905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:07.971995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1034/2068] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:07.982773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.985496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:08.049381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69542a-56d6-49e7-8daa-96b541d15983 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:08.053333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:08.053893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.054378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.054576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.065149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:08.067047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:08.076988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1035/2069] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:08.664105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:08.664949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:08.679898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8dae4-9968-4cfa-9ca1-74867d7db9c6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:08.686133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:08.687085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.687374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.687708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] PATH_INFO: `/v3/projects/fa3a875da4224d3b8d2f61d85ecb7975` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.689944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa3a875da4224d3b8d2f61d85ecb7975)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:08.692022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:08.830418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa3a875da4224d3b8d2f61d85ecb7975', 'request_id': 'req-40d87990-4bfe-4113-a2aa-6ceb9d21831b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:08.837010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa3a875da4224d3b8d2f61d85ecb7975', 'request_id': 'req-40d87990-4bfe-4113-a2aa-6ceb9d21831b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:08.932733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa3a875da4224d3b8d2f61d85ecb7975', 'request_id': 'req-40d87990-4bfe-4113-a2aa-6ceb9d21831b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:08.945018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] The token cache is being invalidate because project fa3a875da4224d3b8d2f61d85ecb7975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:08.945323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:08.946019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:08.947819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1035/2070] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:08 2022] DELETE /identity/v3/projects/fa3a875da4224d3b8d2f61d85ecb7975 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:08.951916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:08.964993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bcad4-e993-4f63-87de-5a22c6927d1a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:09.037165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:09.049309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:09.050278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:09.050563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:09.050839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] PATH_INFO: `/v3/users/2decbea700934103af4e00ee96e06f87` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:09.053885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2decbea700934103af4e00ee96e06f87)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:09.054671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:09.105746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:09.286724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2decbea700934103af4e00ee96e06f87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:09.299011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2decbea700934103af4e00ee96e06f87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:09.379115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2decbea700934103af4e00ee96e06f87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:09.379326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2decbea700934103af4e00ee96e06f87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:09.379958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2cbb4ba-a6b0-4c4a-a703-0603b7525104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2decbea700934103af4e00ee96e06f87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:09.397032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1036/2071] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:08 2022] DELETE /identity/v3/users/2decbea700934103af4e00ee96e06f87 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:09.935406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:09.940701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:09.960340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d87990-4bfe-4113-a2aa-6ceb9d21831b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:10.053027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:10.053027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:10.053027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:10.053027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] PATH_INFO: `/v3/projects/b102678e111249acb5f9b12c2ffa01ad` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:10.053444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b102678e111249acb5f9b12c2ffa01ad)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:10.053444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:10.248805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:10.258130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b102678e111249acb5f9b12c2ffa01ad', 'request_id': 'req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:10.264369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b102678e111249acb5f9b12c2ffa01ad', 'request_id': 'req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:10.339750 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b102678e111249acb5f9b12c2ffa01ad', 'request_id': 'req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:10.352599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] The token cache is being invalidate because project b102678e111249acb5f9b12c2ffa01ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:10.352931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:10.353635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d65ce335-f8f8-4e70-98f8-14c06e78e6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:10.355316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1036/2072] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:09 2022] DELETE /identity/v3/projects/b102678e111249acb5f9b12c2ffa01ad => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:10.361125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ab16e-2d72-497e-aceb-ce65ccfff408 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:10.361747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ab16e-2d72-497e-aceb-ce65ccfff408 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:10.361747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ab16e-2d72-497e-aceb-ce65ccfff408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:10.380234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-980ab16e-2d72-497e-aceb-ce65ccfff408 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:10.381486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-980ab16e-2d72-497e-aceb-ce65ccfff408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:10.452713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1037/2073] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:10.461116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9ce37-2fb7-4e60-9b3f-5b88a9d71a39 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:10.461405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9ce37-2fb7-4e60-9b3f-5b88a9d71a39 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:10.461768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9ce37-2fb7-4e60-9b3f-5b88a9d71a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:10.482945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-96b9ce37-2fb7-4e60-9b3f-5b88a9d71a39 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:10.484171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-96b9ce37-2fb7-4e60-9b3f-5b88a9d71a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:10.493815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1037/2074] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:10.505524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ab16e-2d72-497e-aceb-ce65ccfff408 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:10.523502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ab16e-2d72-497e-aceb-ce65ccfff408 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:10.523502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:10.523502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:10.523502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:10.523952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:10.523952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:10.523952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:10.525484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:10.539032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1038/2075] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:10.544098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9ce37-2fb7-4e60-9b3f-5b88a9d71a39 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:10.556823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:10.559015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9ce37-2fb7-4e60-9b3f-5b88a9d71a39 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:10.565568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:10.566387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:10.566668 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:10.566942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:10.567656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:10.569596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:10.620704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:10.698599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1038/2076] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:10 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:10.703711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:10.713480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1e64a-a317-468b-9f37-431bb0fde5e8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:10.784439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:10.785145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:10.785429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:10.785704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:10.786446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:10.788409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:10.941944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1039/2077] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:10 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:10.947490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:10.965040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54551-4435-408b-b367-6ea877d6784c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:10.971431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:10.972190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:10.972482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:10.972756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:10.973536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:10.975157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:10.986148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1039/2078] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:10.990575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:11.003699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76aaf-02c7-4149-9c7b-1ce0c6b6278d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:11.008606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514887e-5d9b-4ca5-b429-2c953a806af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:11.009317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514887e-5d9b-4ca5-b429-2c953a806af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:11.009604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514887e-5d9b-4ca5-b429-2c953a806af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:11.015218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514887e-5d9b-4ca5-b429-2c953a806af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:11.015218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514887e-5d9b-4ca5-b429-2c953a806af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:11.015218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514887e-5d9b-4ca5-b429-2c953a806af1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:11.025227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1040/2079] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:11.030101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:11.041915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e889-d4b1-4b32-9c51-847084c37925 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:11.047173 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:11.048235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:11.048593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:11.048991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] PATH_INFO: `/v3/projects/18868073b0fe4d9a9e1b47175aee88c9/users/152e32916bab42319af3ba35de3744ab/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:11.067338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18868073b0fe4d9a9e1b47175aee88c9, user_id=152e32916bab42319af3ba35de3744ab, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:11.070408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:11.162787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:11.169886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1040/2080] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:11 2022] PUT /identity/v3/projects/18868073b0fe4d9a9e1b47175aee88c9/users/152e32916bab42319af3ba35de3744ab/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:11.181627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43146f-2c4f-4a19-8782-1ce4ba4c1896 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:11.181960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43146f-2c4f-4a19-8782-1ce4ba4c1896 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:11.182257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43146f-2c4f-4a19-8782-1ce4ba4c1896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:11.212939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e43146f-2c4f-4a19-8782-1ce4ba4c1896 None None] MFA Rules not processed for user `152e32916bab42319af3ba35de3744ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:11.214197 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e43146f-2c4f-4a19-8782-1ce4ba4c1896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:11.271648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:11.292173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1041/2081] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:11.296660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:11.367400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1e507-21c5-4d9a-abc9-a594a2be4033 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:11.372714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:11.373495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:11.373814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:11.374092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:11.374802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:11.377379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:11.537567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1041/2082] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:11 2022] POST /identity/v3/projects => generated 387 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:11.542373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43146f-2c4f-4a19-8782-1ce4ba4c1896 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:11.558191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43146f-2c4f-4a19-8782-1ce4ba4c1896 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:11.627592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:11.628294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:11.628635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:11.628959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:11.629760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:11.631770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:11.778662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1042/2083] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:11 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:11.783275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:11.801363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfc9b7-c8ba-486d-9ded-ff65d12aba21 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:11.807923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:11.808635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:11.808948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:11.809253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:11.809985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:11.811617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:11.823172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1042/2084] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:11.830768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:11.841551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567435f-6f53-4d39-996b-65e4adc3d449 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:11.848069 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:11.849029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:11.849314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:11.849588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] PATH_INFO: `/v3/projects/4f0425c281c544aca65dd7ca02e564ba/users/c353bf19c24b4c6282ae25bdcbe9082f/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:11.868816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0425c281c544aca65dd7ca02e564ba, user_id=c353bf19c24b4c6282ae25bdcbe9082f, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:11.871800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:12.041459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1043/2085] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:11 2022] PUT /identity/v3/projects/4f0425c281c544aca65dd7ca02e564ba/users/c353bf19c24b4c6282ae25bdcbe9082f/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:49:12.045445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:12.055603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:12.057673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1424c4c-6f8b-4d26-a23f-3d725c05c241 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:12.124402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfd132-43fa-4803-9e57-c98c8b95d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:12.125159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfd132-43fa-4803-9e57-c98c8b95d5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:12.125447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfd132-43fa-4803-9e57-c98c8b95d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:12.125726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfd132-43fa-4803-9e57-c98c8b95d5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:12.126563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfd132-43fa-4803-9e57-c98c8b95d5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:12.128190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfd132-43fa-4803-9e57-c98c8b95d5c5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:12.138726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1043/2086] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:12.143105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:12.155117 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86530911-ec32-4498-a5ad-9c3aaf5e09ef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:12.160588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:12.161652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:12.161941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:12.162217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] PATH_INFO: `/v3/projects/4f0425c281c544aca65dd7ca02e564ba/users/c353bf19c24b4c6282ae25bdcbe9082f/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:12.164206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0425c281c544aca65dd7ca02e564ba, user_id=c353bf19c24b4c6282ae25bdcbe9082f, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:12.166988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:12.178526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1044/2087] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:12 2022] PUT /identity/v3/projects/4f0425c281c544aca65dd7ca02e564ba/users/c353bf19c24b4c6282ae25bdcbe9082f/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:12.189476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018f83f-03c1-41ea-8ca9-35f7f813a75a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:12.190897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018f83f-03c1-41ea-8ca9-35f7f813a75a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:12.190897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018f83f-03c1-41ea-8ca9-35f7f813a75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:12.217571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:12.221729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f018f83f-03c1-41ea-8ca9-35f7f813a75a None None] MFA Rules not processed for user `c353bf19c24b4c6282ae25bdcbe9082f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:12.223011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f018f83f-03c1-41ea-8ca9-35f7f813a75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:12.301898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1044/2088] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:12.313894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:12.316375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:12.347858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:12.386936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:12.396742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd9c89-911c-43f9-9b76-58fab9169a1b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:12.400554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65693f28-b906-4edc-957f-024c0ebeed5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:12.401301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693f28-b906-4edc-957f-024c0ebeed5d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:12.401587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693f28-b906-4edc-957f-024c0ebeed5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:12.401868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693f28-b906-4edc-957f-024c0ebeed5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:12.413354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65693f28-b906-4edc-957f-024c0ebeed5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:12.414836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65693f28-b906-4edc-957f-024c0ebeed5d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:12.428185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1045/2089] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:49:12.629154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018f83f-03c1-41ea-8ca9-35f7f813a75a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:12.687080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:12.701018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018f83f-03c1-41ea-8ca9-35f7f813a75a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:12.705915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:12.706801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:12.707084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:12.707359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] PATH_INFO: `/v3/users/152e32916bab42319af3ba35de3744ab` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:12.708551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=152e32916bab42319af3ba35de3744ab)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:12.710624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:12.757366 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:13.024892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152e32916bab42319af3ba35de3744ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:13.034167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152e32916bab42319af3ba35de3744ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:13.049354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152e32916bab42319af3ba35de3744ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:13.145383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152e32916bab42319af3ba35de3744ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:13.145975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152e32916bab42319af3ba35de3744ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:13.155489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1045/2090] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:12 2022] DELETE /identity/v3/users/152e32916bab42319af3ba35de3744ab => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:13.162400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:13.163097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:13.163444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:13.183815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:13.185264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:13.255118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1046/2091] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:13.266071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:13.267595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:13.340566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54b635-d489-4e0a-bedc-ee05a6e511ea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:13.353015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:13.353813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:13.354140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:13.354485 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:13.371584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:13.374335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:13.386615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1046/2092] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:49:13.608954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:13.679291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:14.090239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:14.091762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:14.107789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10036636-1301-4fd2-a8f6-9e04d46a1995 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:14.115157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:14.116222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:14.116645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:14.117049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] PATH_INFO: `/v3/projects/18868073b0fe4d9a9e1b47175aee88c9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:14.118987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18868073b0fe4d9a9e1b47175aee88c9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:14.121531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:14.324235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18868073b0fe4d9a9e1b47175aee88c9', 'request_id': 'req-66af89a5-aeea-4f45-b918-99073bfb0f40'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:14.477489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18868073b0fe4d9a9e1b47175aee88c9', 'request_id': 'req-66af89a5-aeea-4f45-b918-99073bfb0f40'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:14.504150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18868073b0fe4d9a9e1b47175aee88c9', 'request_id': 'req-66af89a5-aeea-4f45-b918-99073bfb0f40'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:14.520937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] The token cache is being invalidate because project 18868073b0fe4d9a9e1b47175aee88c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:14.522111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:14.522953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:14.525123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1047/2093] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:14 2022] DELETE /identity/v3/projects/18868073b0fe4d9a9e1b47175aee88c9 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:14.530805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:14.551474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8a22-732e-4981-bfb0-5fad7b73f833 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:14.644880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:14.645917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:14.646408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:14.646767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] PATH_INFO: `/v3/users/c353bf19c24b4c6282ae25bdcbe9082f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:14.648208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c353bf19c24b4c6282ae25bdcbe9082f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:14.650492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:14.780599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:14.929109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:14.946468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c353bf19c24b4c6282ae25bdcbe9082f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:14.963648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c353bf19c24b4c6282ae25bdcbe9082f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:14.987649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c353bf19c24b4c6282ae25bdcbe9082f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:15.135078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c353bf19c24b4c6282ae25bdcbe9082f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:15.137659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e503868e-3bc7-49cd-8d97-feaac6cd8a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c353bf19c24b4c6282ae25bdcbe9082f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:15.151744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1047/2094] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:14 2022] DELETE /identity/v3/users/c353bf19c24b4c6282ae25bdcbe9082f => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:15.372604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:15.473495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:15.773204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:15.775855 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:15.801168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af89a5-aeea-4f45-b918-99073bfb0f40 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:15.885613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:15.886457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:15.886743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:15.887122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] PATH_INFO: `/v3/projects/4f0425c281c544aca65dd7ca02e564ba` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:15.888358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0425c281c544aca65dd7ca02e564ba)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:15.890372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:16.069836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0425c281c544aca65dd7ca02e564ba', 'request_id': 'req-4550ec3c-90da-4f4a-a170-375273811417'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:16.168335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0425c281c544aca65dd7ca02e564ba', 'request_id': 'req-4550ec3c-90da-4f4a-a170-375273811417'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:16.181701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0425c281c544aca65dd7ca02e564ba', 'request_id': 'req-4550ec3c-90da-4f4a-a170-375273811417'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:16.203682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] The token cache is being invalidate because project 4f0425c281c544aca65dd7ca02e564ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:16.204004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:16.205358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4550ec3c-90da-4f4a-a170-375273811417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:16.208594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1048/2095] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:15 2022] DELETE /identity/v3/projects/4f0425c281c544aca65dd7ca02e564ba => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:16.218197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4616d7-f873-4227-9f84-9d7e12f940be None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.218197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4616d7-f873-4227-9f84-9d7e12f940be None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.218197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4616d7-f873-4227-9f84-9d7e12f940be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.239993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2b4616d7-f873-4227-9f84-9d7e12f940be None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:16.241886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2b4616d7-f873-4227-9f84-9d7e12f940be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:16.341705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1048/2096] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:16.343572 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81e92-5c54-4328-b2e1-555b4d7d3948 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.343923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81e92-5c54-4328-b2e1-555b4d7d3948 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.344570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81e92-5c54-4328-b2e1-555b4d7d3948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.364840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-57e81e92-5c54-4328-b2e1-555b4d7d3948 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:16.366073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-57e81e92-5c54-4328-b2e1-555b4d7d3948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:16.374666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1049/2097] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:16.380488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4616d7-f873-4227-9f84-9d7e12f940be None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:16.394419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4616d7-f873-4227-9f84-9d7e12f940be None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:16.399291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:16.399969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.400256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.400592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.401375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:16.401697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:16.403331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:16.417359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1049/2098] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:16.422443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81e92-5c54-4328-b2e1-555b4d7d3948 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:16.432350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81e92-5c54-4328-b2e1-555b4d7d3948 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:16.438106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:16.439211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.439532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.439814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.440532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:16.442680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:16.567159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1050/2099] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:16 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:16.572211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:16.584330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec33ec2-eeeb-49d0-a732-4bb2fdfd213c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:16.654982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:16.655784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.656103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.656420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.657444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:16.659817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:16.850719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1050/2100] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:16 2022] POST /identity/v3/users => generated 292 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:16.855096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:16.861792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:16.867560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04227330-73ce-4fd5-8ccc-6ed2f38d4de8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:16.872263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:16.872976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.873258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.873531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.877761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:16.877761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:16.888628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1051/2101] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:16.893092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:16.904637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65ddeb-2066-40e1-ab9f-b32813cec7f1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:16.913130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:16.913130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.913130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.913130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.913788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:16.916709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:16.931322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1051/2102] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:16.937303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:16.951193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77677e-fcf7-4050-b6aa-fa62698b7ef5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:16.957133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba28c98-8863-474b-bdf0-abd883d316f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:16.958067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28c98-8863-474b-bdf0-abd883d316f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:16.958364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28c98-8863-474b-bdf0-abd883d316f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:16.958636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28c98-8863-474b-bdf0-abd883d316f8 admin admin] PATH_INFO: `/v3/projects/aab26869c05545acb7882a9481854927/users/4a8bcabd9df647879181ef6f71ddc6a7/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:16.993348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba28c98-8863-474b-bdf0-abd883d316f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab26869c05545acb7882a9481854927, user_id=4a8bcabd9df647879181ef6f71ddc6a7, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:16.993805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:16.995633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:16.996482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba28c98-8863-474b-bdf0-abd883d316f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:17.015626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb602f-de98-4a4a-9bb9-803cc31dcb3d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:17.021329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:17.022362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:17.022679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:17.022964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] PATH_INFO: `/v3/users/169b8104259b47958ec674b98794bd26` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:17.024401 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=169b8104259b47958ec674b98794bd26)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:17.026472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:17.107191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1052/2103] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:16 2022] PUT /identity/v3/projects/aab26869c05545acb7882a9481854927/users/4a8bcabd9df647879181ef6f71ddc6a7/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:17.119574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368b532-3b5d-46a8-b0a6-3f6a77911677 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:17.119866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368b532-3b5d-46a8-b0a6-3f6a77911677 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:17.120149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368b532-3b5d-46a8-b0a6-3f6a77911677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:17.152217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a368b532-3b5d-46a8-b0a6-3f6a77911677 None None] MFA Rules not processed for user `4a8bcabd9df647879181ef6f71ddc6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:17.153568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a368b532-3b5d-46a8-b0a6-3f6a77911677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:17.193900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:17.258923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1053/2104] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:17.264669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368b532-3b5d-46a8-b0a6-3f6a77911677 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:17.377450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368b532-3b5d-46a8-b0a6-3f6a77911677 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:17.383960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:17.384909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:17.385395 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:17.385824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:17.386724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:17.388801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '169b8104259b47958ec674b98794bd26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:17.389703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:17.409488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '169b8104259b47958ec674b98794bd26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:17.432960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '169b8104259b47958ec674b98794bd26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:17.538273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '169b8104259b47958ec674b98794bd26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:17.539406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-186f689c-05c5-4946-9b96-dedf8a0703c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '169b8104259b47958ec674b98794bd26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:17.545851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1054/2105] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:17 2022] POST /identity/v3/projects => generated 381 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:17.552006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:17.557760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:17.569395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1052/2106] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:16 2022] DELETE /identity/v3/users/169b8104259b47958ec674b98794bd26 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:17.572742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:17.573271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:17.574025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:17.615638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:17.618067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:17.718423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe03b37-6722-4512-97d4-c05080686764 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:17.744077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1053/2107] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:17.756125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:17.758111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:17.760472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:17.830840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:17.831562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:17.831849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:17.832126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:17.832927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:17.836038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:17.910193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46593ebf-d759-4b94-b00a-96e7d44ca75c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:17.914867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:17.915715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:17.916232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:17.916623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:17.928879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:17.930987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:17.944585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1054/2108] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:17.958561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1055/2109] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:17 2022] POST /identity/v3/users => generated 297 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:17.962186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:17.973255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc906dbe-54c2-4c11-9393-0136e4385d87 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:17.983696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:17.983696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:17.983696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:17.983696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:17.984243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:17.984918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:17.997512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1055/2110] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:18.002575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:18.013870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24995a-9e65-42da-96d9-38f2bcace42a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:18.020441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:18.021442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:18.021714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:18.021994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] PATH_INFO: `/v3/projects/a187841055cc4cb48116febe30ff6e1a/users/fbe8807bf08c45dda5f2d630ae67973f/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:18.040013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a187841055cc4cb48116febe30ff6e1a, user_id=fbe8807bf08c45dda5f2d630ae67973f, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:18.042918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:18.167144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1056/2111] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:18 2022] PUT /identity/v3/projects/a187841055cc4cb48116febe30ff6e1a/users/fbe8807bf08c45dda5f2d630ae67973f/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:18.171292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:18.182473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5365526-674d-4d7b-966f-3169bbb5606b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:18.223376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:18.249701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d309d17-e00f-4d6b-a19a-3f112712532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:18.250731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d309d17-e00f-4d6b-a19a-3f112712532b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:18.250731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d309d17-e00f-4d6b-a19a-3f112712532b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:18.251127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d309d17-e00f-4d6b-a19a-3f112712532b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:18.251972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d309d17-e00f-4d6b-a19a-3f112712532b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:18.253664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d309d17-e00f-4d6b-a19a-3f112712532b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:18.266869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1056/2112] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:18.270608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:18.283037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cfe71-ac95-4e7e-ad96-9dab7802c8b7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:18.288911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:18.294920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:18.294920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:18.294920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] PATH_INFO: `/v3/projects/a187841055cc4cb48116febe30ff6e1a/users/fbe8807bf08c45dda5f2d630ae67973f/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:18.294920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a187841055cc4cb48116febe30ff6e1a, user_id=fbe8807bf08c45dda5f2d630ae67973f, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:18.296321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:18.335162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1057/2113] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:18 2022] PUT /identity/v3/projects/a187841055cc4cb48116febe30ff6e1a/users/fbe8807bf08c45dda5f2d630ae67973f/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:18.347320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:18.347616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:18.347888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:18.384448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] MFA Rules not processed for user `fbe8807bf08c45dda5f2d630ae67973f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:18.385867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:18.504796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1057/2114] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:18.519782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:18.523751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:18.602370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e92300-a2b9-467f-807a-4a0581d960fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:18.605701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:18.606450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:18.607221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:18.607529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:18.617847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:18.619309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:18.630651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1058/2115] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:18.889472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:18.892769 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:18.957290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:18.971632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563ae4e-f9d7-47a7-acf7-0f824d8328bf service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:18.978471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c45c2-fab3-4d89-9195-36d223008c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:18.979447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c45c2-fab3-4d89-9195-36d223008c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:18.979725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c45c2-fab3-4d89-9195-36d223008c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:18.980183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c45c2-fab3-4d89-9195-36d223008c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:18.983941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7f9c45c2-fab3-4d89-9195-36d223008c14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:19.012406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2a698-9e29-465b-8be4-248fe995c125 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:19.018328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:19.019307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.019704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.020082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] PATH_INFO: `/v3/projects/455909c460574d7c9621fa9802970c48` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.021464 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=455909c460574d7c9621fa9802970c48)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:19.024297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:19.060854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:19.079707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c45c2-fab3-4d89-9195-36d223008c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:19.082610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c45c2-fab3-4d89-9195-36d223008c14 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:19.099912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1059/2116] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:49:19.226142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455909c460574d7c9621fa9802970c48', 'request_id': 'req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:19.233847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455909c460574d7c9621fa9802970c48', 'request_id': 'req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:19.393126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455909c460574d7c9621fa9802970c48', 'request_id': 'req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:19.406061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] The token cache is being invalidate because project 455909c460574d7c9621fa9802970c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:19.406607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:19.410634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b42ba87-cff9-4140-98a4-a2f59dc86bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:19.416794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1058/2117] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:18 2022] DELETE /identity/v3/projects/455909c460574d7c9621fa9802970c48 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:19.423834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de0287-c46b-416d-aeae-47eac044c536 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.424122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de0287-c46b-416d-aeae-47eac044c536 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.424416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de0287-c46b-416d-aeae-47eac044c536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.444371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-83de0287-c46b-416d-aeae-47eac044c536 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:19.445665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-83de0287-c46b-416d-aeae-47eac044c536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:19.512304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1060/2118] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:19.521263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f9308-d922-4d39-9362-86640a04c1e9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.521783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f9308-d922-4d39-9362-86640a04c1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.521918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f9308-d922-4d39-9362-86640a04c1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.542397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2d0f9308-d922-4d39-9362-86640a04c1e9 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:19.543994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d0f9308-d922-4d39-9362-86640a04c1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:19.553944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1059/2119] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:19.559414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de0287-c46b-416d-aeae-47eac044c536 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:19.574238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de0287-c46b-416d-aeae-47eac044c536 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:19.581696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:19.582770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.583305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.583972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.585914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:19.585914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:19.588082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:19.618029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1061/2120] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:19.624449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f9308-d922-4d39-9362-86640a04c1e9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:19.639371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f9308-d922-4d39-9362-86640a04c1e9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:19.647507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:19.648640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.649209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.650795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.650956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:19.654640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:19.840896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1060/2121] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:19 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:19.848542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:19.857939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:19.861850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:19.863286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194b322-5954-46f7-bcc7-c2bd0d24ed78 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:19.973196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:19.975074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:19.976134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.976581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.977323 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.978274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:19.980832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:19.986006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f75e5b-a68b-4b90-bfce-4079bba44d40 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:19.993757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:19.994719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.995157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.995491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] PATH_INFO: `/v3/users/4a8bcabd9df647879181ef6f71ddc6a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.996979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8bcabd9df647879181ef6f71ddc6a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:19.999144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:20.094332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1062/2122] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:19 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:20.099238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:20.110979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e684261-fd9c-412a-9d7e-cb1f1db8c592 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:20.120973 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:20.122582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.122582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.122582 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.122952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:20.124628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:20.156205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1063/2123] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:49:20.158264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:20.184566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b258b-4ea7-45c0-abb1-e599301fad03 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:20.191771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622629ff-8d5f-4dd2-92f9-078f25d2c674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:20.192466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622629ff-8d5f-4dd2-92f9-078f25d2c674 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.192754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622629ff-8d5f-4dd2-92f9-078f25d2c674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.193071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622629ff-8d5f-4dd2-92f9-078f25d2c674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.193930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622629ff-8d5f-4dd2-92f9-078f25d2c674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:20.195665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622629ff-8d5f-4dd2-92f9-078f25d2c674 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:20.208622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1064/2124] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:20.210036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837723f-8efe-483f-a924-da108475fe49 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.211045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837723f-8efe-483f-a924-da108475fe49 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.211045 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837723f-8efe-483f-a924-da108475fe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.242038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4837723f-8efe-483f-a924-da108475fe49 None None] MFA Rules not processed for user `49e86a3f10b64ef38664013e197782c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:20.243697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4837723f-8efe-483f-a924-da108475fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:20.309188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8bcabd9df647879181ef6f71ddc6a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.322294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1065/2125] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:20.323433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837723f-8efe-483f-a924-da108475fe49 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:20.324981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8bcabd9df647879181ef6f71ddc6a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.340607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837723f-8efe-483f-a924-da108475fe49 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:20.347227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7484aa-387b-43a2-b819-9e1b32015f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:20.348453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7484aa-387b-43a2-b819-9e1b32015f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.348746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7484aa-387b-43a2-b819-9e1b32015f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.349061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7484aa-387b-43a2-b819-9e1b32015f0d admin admin] PATH_INFO: `/v3/projects/17f436162f3a4bd4b3a94f584e7d8782/users/eb483b1aee944da8910dd7468a0ef4ac/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.351141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8bcabd9df647879181ef6f71ddc6a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.373138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:20.378853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7484aa-387b-43a2-b819-9e1b32015f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f436162f3a4bd4b3a94f584e7d8782, user_id=eb483b1aee944da8910dd7468a0ef4ac, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:20.382482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7484aa-387b-43a2-b819-9e1b32015f0d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:20.488674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8bcabd9df647879181ef6f71ddc6a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.489444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-087a138c-07d5-4240-8f5a-9e6690f63463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8bcabd9df647879181ef6f71ddc6a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.507963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1061/2126] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:19 2022] DELETE /identity/v3/users/4a8bcabd9df647879181ef6f71ddc6a7 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:49:20.512596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.512907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.513192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.533351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:20.534845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:20.538419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1066/2127] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:20 2022] PUT /identity/v3/projects/17f436162f3a4bd4b3a94f584e7d8782/users/eb483b1aee944da8910dd7468a0ef4ac/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:20.543457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.543753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.544065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.568076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:20.569744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:20.614051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1062/2128] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:20.626904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:20.629059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:20.667360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1067/2129] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:20.689917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:20.694714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:20.716727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f058a4-c993-4a76-b6b0-e949025c1d83 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:20.725714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:20.727006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.727578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.727776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027c3b3-3d38-423b-976e-e79c85cc2f54 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:20.728202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.735577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:20.736906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.737406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.738200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.741206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:20.742702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:20.755764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1063/2130] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:49:20.758295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:20.761473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:20.782280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1068/2131] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:21.431880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:21.435123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:21.460376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e970db9-cba7-4840-83e6-7e20485873a1 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:21.466473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:21.467633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:21.468096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:21.469325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] PATH_INFO: `/v3/projects/aab26869c05545acb7882a9481854927` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:21.470348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab26869c05545acb7882a9481854927)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:21.483504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:21.591863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:21.711336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab26869c05545acb7882a9481854927', 'request_id': 'req-41b5a4f3-190f-4125-889b-3a028cfd8e08'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:21.717952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab26869c05545acb7882a9481854927', 'request_id': 'req-41b5a4f3-190f-4125-889b-3a028cfd8e08'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:21.882460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab26869c05545acb7882a9481854927', 'request_id': 'req-41b5a4f3-190f-4125-889b-3a028cfd8e08'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:21.895792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] The token cache is being invalidate because project aab26869c05545acb7882a9481854927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:21.896085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:21.896920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:21.898565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1064/2132] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:21 2022] DELETE /identity/v3/projects/aab26869c05545acb7882a9481854927 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:21.902697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:21.913792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff906ce2-e5f1-453c-89d0-79f5e48d5414 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:22.024532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:22.025478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:22.025766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:22.026044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] PATH_INFO: `/v3/users/fbe8807bf08c45dda5f2d630ae67973f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:22.027355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbe8807bf08c45dda5f2d630ae67973f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:22.029460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:22.350407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbe8807bf08c45dda5f2d630ae67973f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.352099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:22.366116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbe8807bf08c45dda5f2d630ae67973f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.494116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbe8807bf08c45dda5f2d630ae67973f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.494874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbe8807bf08c45dda5f2d630ae67973f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.495613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad2d3a7-c20f-46a6-9170-f070658ee790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbe8807bf08c45dda5f2d630ae67973f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.511886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1069/2133] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:21 2022] DELETE /identity/v3/users/fbe8807bf08c45dda5f2d630ae67973f => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:23.305007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:23.306241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:23.326946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5a4f3-190f-4125-889b-3a028cfd8e08 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:23.428080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:23.429133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:23.429422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:23.429704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] PATH_INFO: `/v3/projects/a187841055cc4cb48116febe30ff6e1a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:23.431135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a187841055cc4cb48116febe30ff6e1a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:23.433408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:23.443797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:23.717654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a187841055cc4cb48116febe30ff6e1a', 'request_id': 'req-311f4c6c-db5d-4325-bd69-e8c37049dd41'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:23.724929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a187841055cc4cb48116febe30ff6e1a', 'request_id': 'req-311f4c6c-db5d-4325-bd69-e8c37049dd41'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:23.996264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a187841055cc4cb48116febe30ff6e1a', 'request_id': 'req-311f4c6c-db5d-4325-bd69-e8c37049dd41'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.012282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] The token cache is being invalidate because project a187841055cc4cb48116febe30ff6e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:24.012282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.012282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-311f4c6c-db5d-4325-bd69-e8c37049dd41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.012901 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1065/2134] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:23 2022] DELETE /identity/v3/projects/a187841055cc4cb48116febe30ff6e1a => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:24.017708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504939d-463b-4cf5-9f2b-3ddff25ed037 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.032248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504939d-463b-4cf5-9f2b-3ddff25ed037 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.032611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504939d-463b-4cf5-9f2b-3ddff25ed037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.058795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4504939d-463b-4cf5-9f2b-3ddff25ed037 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:24.060047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4504939d-463b-4cf5-9f2b-3ddff25ed037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:24.152946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1070/2135] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:24.157358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395625d5-c167-41e3-b959-f9313fbab72d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.163548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395625d5-c167-41e3-b959-f9313fbab72d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.163548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395625d5-c167-41e3-b959-f9313fbab72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.190925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-395625d5-c167-41e3-b959-f9313fbab72d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:24.192514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-395625d5-c167-41e3-b959-f9313fbab72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:24.208539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504939d-463b-4cf5-9f2b-3ddff25ed037 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:24.215771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1066/2136] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:24.221728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504939d-463b-4cf5-9f2b-3ddff25ed037 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.229537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.230465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.230902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.231361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.232292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:24.232762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:24.235018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:24.258180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1071/2137] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:24.259530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395625d5-c167-41e3-b959-f9313fbab72d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:24.271204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395625d5-c167-41e3-b959-f9313fbab72d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.278278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.279133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.279415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.279694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.280543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:24.282803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:24.413631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1067/2138] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:24 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:24.414274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:24.433899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d52c-f8d7-4b9b-8a98-e02f87ab494c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.557820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.558486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.558776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.559053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.559762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:24.561979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:24.832041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1072/2139] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:24 2022] POST /identity/v3/users => generated 292 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:24.836359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:24.850197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00591854-928c-41ac-bceb-4ceedccdfe10 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.869224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.871365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.873025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.873333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.877670 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:24.880262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:24.904752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1068/2140] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:24.904881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:24.916194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f659-ed09-45c7-b1e2-769200986186 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.923953 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574c8f5-2c16-47f5-bd24-3925a9b07b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.926649 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c8f5-2c16-47f5-bd24-3925a9b07b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.927155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c8f5-2c16-47f5-bd24-3925a9b07b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.927514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c8f5-2c16-47f5-bd24-3925a9b07b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.931455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574c8f5-2c16-47f5-bd24-3925a9b07b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:24.933513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574c8f5-2c16-47f5-bd24-3925a9b07b22 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:24.946988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1073/2141] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:24.951555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:24.974414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284954b7-9ba1-44f2-b412-1e09565e869e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.987451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.993725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.994204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.994587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] PATH_INFO: `/v3/projects/64da1223e8db432099c5324e24522efb/users/9b18c166f3c34303ac398754b5dc7090/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:25.048011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64da1223e8db432099c5324e24522efb, user_id=9b18c166f3c34303ac398754b5dc7090, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:25.053063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:25.187997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1069/2142] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:24 2022] PUT /identity/v3/projects/64da1223e8db432099c5324e24522efb/users/9b18c166f3c34303ac398754b5dc7090/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:25.200351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea34e5-3c32-4215-849c-7209fa7b3a87 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:25.200930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea34e5-3c32-4215-849c-7209fa7b3a87 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:25.201216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea34e5-3c32-4215-849c-7209fa7b3a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:25.260211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-42ea34e5-3c32-4215-849c-7209fa7b3a87 None None] MFA Rules not processed for user `9b18c166f3c34303ac398754b5dc7090`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:25.261675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-42ea34e5-3c32-4215-849c-7209fa7b3a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:25.289802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:25.376659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1074/2143] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:25.381366 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:25.506395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957643c6-6eb4-4a1e-ae1c-6e93fe335c22 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:25.512651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:25.513855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:25.514364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:25.514862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:25.515909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:25.518902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:25.721895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1070/2144] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:25 2022] POST /identity/v3/projects => generated 385 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:25.722395 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea34e5-3c32-4215-849c-7209fa7b3a87 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:25.747415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea34e5-3c32-4215-849c-7209fa7b3a87 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:25.858709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:25.859941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:25.860233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:25.860547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:25.861263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:25.863375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.110647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1075/2145] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:25 2022] POST /identity/v3/users => generated 299 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:26.136564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.149712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0812242-864c-473f-af57-da99fb56da49 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.154638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.155665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.156874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.157220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.157980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.159651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.182459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1071/2146] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:26.187118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.199314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e8669-f323-470a-b891-f5e2bf7cc299 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.204712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.219196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.219196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.219196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] PATH_INFO: `/v3/projects/fd464132becd4d9a877a682fa0fa73fc/users/46e53572cb514839ba46f2a530c58ec7/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.227396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd464132becd4d9a877a682fa0fa73fc, user_id=46e53572cb514839ba46f2a530c58ec7, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.230533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.494715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1076/2147] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:26 2022] PUT /identity/v3/projects/fd464132becd4d9a877a682fa0fa73fc/users/46e53572cb514839ba46f2a530c58ec7/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:26.499099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.514626 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d0b06-f00b-4c15-b9e8-0f65af2871cc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.586117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966dfcf-0ff8-46c2-976b-1cc853d6c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.586791 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966dfcf-0ff8-46c2-976b-1cc853d6c905 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.587074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966dfcf-0ff8-46c2-976b-1cc853d6c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.587349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966dfcf-0ff8-46c2-976b-1cc853d6c905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.588271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966dfcf-0ff8-46c2-976b-1cc853d6c905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.590007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966dfcf-0ff8-46c2-976b-1cc853d6c905 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.602197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1072/2148] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:49:26.606514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.618148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a7ebe-8ff4-452c-864d-9b78826ab956 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.624595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.626193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.632232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.632232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] PATH_INFO: `/v3/projects/fd464132becd4d9a877a682fa0fa73fc/users/46e53572cb514839ba46f2a530c58ec7/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.632232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd464132becd4d9a877a682fa0fa73fc, user_id=46e53572cb514839ba46f2a530c58ec7, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.632232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.661839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1077/2149] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:26 2022] PUT /identity/v3/projects/fd464132becd4d9a877a682fa0fa73fc/users/46e53572cb514839ba46f2a530c58ec7/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:26.662748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70bcd-20ab-4f08-883a-ab1b4c221b21 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.663056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70bcd-20ab-4f08-883a-ab1b4c221b21 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.663659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70bcd-20ab-4f08-883a-ab1b4c221b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.699003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e8c70bcd-20ab-4f08-883a-ab1b4c221b21 None None] MFA Rules not processed for user `46e53572cb514839ba46f2a530c58ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:26.700327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e8c70bcd-20ab-4f08-883a-ab1b4c221b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:26.792952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1073/2150] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:26.807528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.810180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:26.916621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f85cf-a1db-4393-8f88-adf1311332ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.924122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034235ef-b02a-430d-9d95-a2406e588084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.925268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034235ef-b02a-430d-9d95-a2406e588084 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.925809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034235ef-b02a-430d-9d95-a2406e588084 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.926308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034235ef-b02a-430d-9d95-a2406e588084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.012096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034235ef-b02a-430d-9d95-a2406e588084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:27.014467 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034235ef-b02a-430d-9d95-a2406e588084 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.054951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1078/2151] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:27.271406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:29.926837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:30.283505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071d26-def5-4c6f-a9f6-b1bb18f47cbc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:30.284152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071d26-def5-4c6f-a9f6-b1bb18f47cbc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:30.284152 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071d26-def5-4c6f-a9f6-b1bb18f47cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:30.306316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-78071d26-def5-4c6f-a9f6-b1bb18f47cbc None None] MFA Rules not processed for user `e7a738986f87470681c6a578599a089f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:30.307862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-78071d26-def5-4c6f-a9f6-b1bb18f47cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:30.545879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1074/2152] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:31.037979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:31.038654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:31.038892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:31.059662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] MFA Rules not processed for user `3954fd4abf6b4c3a82280c762292a3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:31.061596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:31.153742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1079/2153] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:33.139404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc53c56-bb74-4f97-800c-5ad97986a9f0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:33.144639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc53c56-bb74-4f97-800c-5ad97986a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:33.146372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc53c56-bb74-4f97-800c-5ad97986a9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:33.209148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bfc53c56-bb74-4f97-800c-5ad97986a9f0 None None] MFA Rules not processed for user `eb483b1aee944da8910dd7468a0ef4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:33.211324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bfc53c56-bb74-4f97-800c-5ad97986a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:33.250889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:33.325788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1075/2154] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:33.332295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:33.335283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:33.455586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd8646-be9a-4e3d-b247-14490f567a67 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:33.464255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:33.465242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:33.465698 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:33.465977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:33.466891 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:33.470873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:33.628636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1080/2155] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:33 2022] POST /identity/v3/projects => generated 379 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:33.633844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc53c56-bb74-4f97-800c-5ad97986a9f0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:33.641403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:33.643540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:33.667305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc53c56-bb74-4f97-800c-5ad97986a9f0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:33.758267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e127db-7f9d-4545-9ea7-5915d20532ea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:33.768458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:33.771560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:33.773381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:33.773882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:33.815314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:33.918995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:33.920193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:33.920759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:33.922737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:33.924219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:33.935650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.015131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:34.017594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.048221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1081/2156] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:34.109873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1076/2157] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:33 2022] POST /identity/v3/users => generated 296 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:34.110061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:34.143935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a067b-b92b-440b-b1be-97fa7cc5ca26 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:34.161619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:34.162560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.165106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.165562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.166425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:34.168179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.196909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1082/2158] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:49:34.201278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:34.219808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04fb95-f1cf-4b79-b45e-2e6bbf0f7bf3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:34.246550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:34.247575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.247981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.248324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] PATH_INFO: `/v3/projects/78719e43f24b499e8647f50db6771cba/users/2e37b061ecdb4577b13e3cf2615d29d4/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.328112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78719e43f24b499e8647f50db6771cba, user_id=2e37b061ecdb4577b13e3cf2615d29d4, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:34.331324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.524912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1077/2159] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:34 2022] PUT /identity/v3/projects/78719e43f24b499e8647f50db6771cba/users/2e37b061ecdb4577b13e3cf2615d29d4/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:34.529286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:34.541500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a940ec3-6c59-4433-ae55-702b446b30a1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:34.630579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:34.631875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.632353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.633031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.634004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:34.635916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.648548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1083/2160] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:49:34.665674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:34.677505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0213fe-7f98-43c1-b252-425bdd39f75f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:34.682992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964c5e9-35d5-430f-8c52-743e41c64576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:34.684047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c5e9-35d5-430f-8c52-743e41c64576 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.684344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c5e9-35d5-430f-8c52-743e41c64576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.684632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c5e9-35d5-430f-8c52-743e41c64576 admin admin] PATH_INFO: `/v3/projects/78719e43f24b499e8647f50db6771cba/users/2e37b061ecdb4577b13e3cf2615d29d4/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.687384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964c5e9-35d5-430f-8c52-743e41c64576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78719e43f24b499e8647f50db6771cba, user_id=2e37b061ecdb4577b13e3cf2615d29d4, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:34.690564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964c5e9-35d5-430f-8c52-743e41c64576 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.704879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1078/2161] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:34 2022] PUT /identity/v3/projects/78719e43f24b499e8647f50db6771cba/users/2e37b061ecdb4577b13e3cf2615d29d4/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:49:41.584282 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.613912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19a74-8a62-4e49-b5b6-241cc489eec6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:41.714399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:41.715591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.715986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.716368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] PATH_INFO: `/v3/users/9b18c166f3c34303ac398754b5dc7090` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.717946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b18c166f3c34303ac398754b5dc7090)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:41.721290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:42.061408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:42.076728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b18c166f3c34303ac398754b5dc7090'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:42.098364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b18c166f3c34303ac398754b5dc7090'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:42.225943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b18c166f3c34303ac398754b5dc7090'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:42.238767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b18c166f3c34303ac398754b5dc7090'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:42.238767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b18c166f3c34303ac398754b5dc7090'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:42.266361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1084/2162] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:41 2022] DELETE /identity/v3/users/9b18c166f3c34303ac398754b5dc7090 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:42.270865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b48661-731d-4525-a46c-5ce0ea4effd1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:42.271877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b48661-731d-4525-a46c-5ce0ea4effd1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:42.272385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b48661-731d-4525-a46c-5ce0ea4effd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:42.296857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a1b48661-731d-4525-a46c-5ce0ea4effd1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:42.298249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1b48661-731d-4525-a46c-5ce0ea4effd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:42.406481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1079/2163] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:42.438545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:42.438545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:42.470243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:42.470788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:42.471270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:42.502702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:42.504757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:42.522519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b32dc-0834-4dd6-b964-dc16fcc2b4de admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:42.527986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:42.528242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:42.529796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:42.529796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:42.529796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:42.542295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:42.544280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:42.560793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1085/2164] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:42.614059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1080/2165] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:49:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:42.690563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:42.692762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:42.943996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13acf75-20f5-4483-9b68-7d1006b4758f service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:42.955444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:42.956613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:42.957283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:42.957831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:42.976023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:42.979533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:43.002315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1086/2166] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:49:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:49:43.657137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:43.659128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:43.715117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36b44-30a9-460c-b0b8-b931c892f3d3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:43.727206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:43.728178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:43.728610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:43.729135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] PATH_INFO: `/v3/projects/64da1223e8db432099c5324e24522efb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:43.730696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=64da1223e8db432099c5324e24522efb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:43.734691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:43.962635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64da1223e8db432099c5324e24522efb', 'request_id': 'req-bc856d73-ea4b-4c73-ab5f-710038a90dca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:43.981522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64da1223e8db432099c5324e24522efb', 'request_id': 'req-bc856d73-ea4b-4c73-ab5f-710038a90dca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:44.212305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64da1223e8db432099c5324e24522efb', 'request_id': 'req-bc856d73-ea4b-4c73-ab5f-710038a90dca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:44.263897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] The token cache is being invalidate because project 64da1223e8db432099c5324e24522efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:44.263986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:44.265058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:44.278642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1081/2167] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:43 2022] DELETE /identity/v3/projects/64da1223e8db432099c5324e24522efb => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:44.279960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:44.322250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44451f0-bd62-4d6f-b96f-339ee143cae3 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:44.406794 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:44.463664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:44.469828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:44.470752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:44.472491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] PATH_INFO: `/v3/users/46e53572cb514839ba46f2a530c58ec7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:44.476796 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e53572cb514839ba46f2a530c58ec7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:44.480937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:45.015867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e53572cb514839ba46f2a530c58ec7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:45.053125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e53572cb514839ba46f2a530c58ec7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:45.190617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e53572cb514839ba46f2a530c58ec7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:45.191758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e53572cb514839ba46f2a530c58ec7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:45.193109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-82ddac1f-1911-4bc0-9065-cdb42a0c2cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e53572cb514839ba46f2a530c58ec7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:45.240085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1087/2168] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:44 2022] DELETE /identity/v3/users/46e53572cb514839ba46f2a530c58ec7 => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:46.027612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:46.029360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:46.105305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc856d73-ea4b-4c73-ab5f-710038a90dca admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:46.256244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:46.259122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:46.259620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:46.260026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] PATH_INFO: `/v3/projects/fd464132becd4d9a877a682fa0fa73fc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:46.262243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd464132becd4d9a877a682fa0fa73fc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:46.265767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:46.534787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:46.538610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd464132becd4d9a877a682fa0fa73fc', 'request_id': 'req-58c1dc1b-a878-4715-b4bc-bcb18582e517'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:46.555578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd464132becd4d9a877a682fa0fa73fc', 'request_id': 'req-58c1dc1b-a878-4715-b4bc-bcb18582e517'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:46.654156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd464132becd4d9a877a682fa0fa73fc', 'request_id': 'req-58c1dc1b-a878-4715-b4bc-bcb18582e517'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:46.676899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] The token cache is being invalidate because project fd464132becd4d9a877a682fa0fa73fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:46.677717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:46.678600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58c1dc1b-a878-4715-b4bc-bcb18582e517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:46.686340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1082/2169] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:46 2022] DELETE /identity/v3/projects/fd464132becd4d9a877a682fa0fa73fc => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:46.695455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79ccaf-3883-4e06-98ec-e74b1ad6de19 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:46.695455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79ccaf-3883-4e06-98ec-e74b1ad6de19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:46.695455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79ccaf-3883-4e06-98ec-e74b1ad6de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:46.726107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9e79ccaf-3883-4e06-98ec-e74b1ad6de19 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:46.726107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9e79ccaf-3883-4e06-98ec-e74b1ad6de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:46.831252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1088/2170] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:46.842326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004389f5-c14b-42d6-a1be-04a95cb36b84 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:46.842629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004389f5-c14b-42d6-a1be-04a95cb36b84 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:46.843397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004389f5-c14b-42d6-a1be-04a95cb36b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:46.877986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-004389f5-c14b-42d6-a1be-04a95cb36b84 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:46.879371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-004389f5-c14b-42d6-a1be-04a95cb36b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:46.895687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1083/2171] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:46.897031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79ccaf-3883-4e06-98ec-e74b1ad6de19 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:46.910477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79ccaf-3883-4e06-98ec-e74b1ad6de19 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:46.919657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:46.920844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:46.921358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:46.921840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:46.922784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:46.923298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:46.925464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:46.939732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1089/2172] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:46.945248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004389f5-c14b-42d6-a1be-04a95cb36b84 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:46.963648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004389f5-c14b-42d6-a1be-04a95cb36b84 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:46.975545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:46.976245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:46.976535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:46.976952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:46.992349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:46.992349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:47.097099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1084/2173] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:46 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:47.101584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:47.117310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc75ba-ba65-4fef-9235-51db8408ed7a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:47.229758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:47.233254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:47.233254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:47.233254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:47.233254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:47.234372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:47.455569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1090/2174] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:47 2022] POST /identity/v3/users => generated 286 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:47.460359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:47.476254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fac4d0-f2c7-4722-8d95-8a9ee329f00e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:47.489791 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:47.490672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:47.491461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:47.492704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:47.492928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:47.497652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:47.514000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1085/2175] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:47.519594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:47.538151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da721f0e-458c-462f-b37e-ed215c6a7d09 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:47.547784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade39bd-0505-465d-bb5e-976842240f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:47.548629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade39bd-0505-465d-bb5e-976842240f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:47.549179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade39bd-0505-465d-bb5e-976842240f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:47.549473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade39bd-0505-465d-bb5e-976842240f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:47.550599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade39bd-0505-465d-bb5e-976842240f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:47.553397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade39bd-0505-465d-bb5e-976842240f0b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:47.573063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:47.586341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1091/2176] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:47.588872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f784a4-e69c-4bca-9e68-bd2b69548e8c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:47.596096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:47.597290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:47.597425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:47.597819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] PATH_INFO: `/v3/projects/7f36b12884cc423988d28fdcbdb1d8f9/users/926d4c4971f1425b98aa5828f802fb21/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:47.629872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f36b12884cc423988d28fdcbdb1d8f9, user_id=926d4c4971f1425b98aa5828f802fb21, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:47.632556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:47.771671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1086/2177] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:47 2022] PUT /identity/v3/projects/7f36b12884cc423988d28fdcbdb1d8f9/users/926d4c4971f1425b98aa5828f802fb21/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:47.774727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea2dca-9dcc-4d55-b258-723f92e13286 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:47.775315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea2dca-9dcc-4d55-b258-723f92e13286 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:47.775315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea2dca-9dcc-4d55-b258-723f92e13286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:47.804423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-76ea2dca-9dcc-4d55-b258-723f92e13286 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:47.806742 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-76ea2dca-9dcc-4d55-b258-723f92e13286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:47.888486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1092/2178] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:47.907813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:47.909859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:48.012560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00def-4eb6-4c99-aff8-e77f7553ceac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:48.018265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:48.019412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:48.020128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:48.020904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:48.036945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:48.039054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:48.054321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1087/2179] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:48.322773 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:49.020584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:49.021287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:49.021287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:49.057133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] MFA Rules not processed for user `2e37b061ecdb4577b13e3cf2615d29d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:49.058391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:49.177092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1093/2180] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:49:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:49.217694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:49.226729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:49.377459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3268442-f1d9-450b-9511-7d86bf24848d service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:49.388367 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ee05f-bb42-44df-9ccd-1ac0ce63c3c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:49.389267 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee05f-bb42-44df-9ccd-1ac0ce63c3c4 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:49.391177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee05f-bb42-44df-9ccd-1ac0ce63c3c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:49.391177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee05f-bb42-44df-9ccd-1ac0ce63c3c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:49.392318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cf1ee05f-bb42-44df-9ccd-1ac0ce63c3c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:49.491020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ee05f-bb42-44df-9ccd-1ac0ce63c3c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:49.493124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ee05f-bb42-44df-9ccd-1ac0ce63c3c4 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:49.522756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1088/2181] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:50.530042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:56.238912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:56.264554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:56.418252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:56.446010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c16ea-67ae-4245-8375-a271e3044f7f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:56.457322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:56.458396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:56.458786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:56.459721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:56.462473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:56.478659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:56.481543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:56.495403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1094/2182] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:02.621581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6c505-a1d9-43de-877f-39e5ee39e894 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:02.622501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6c505-a1d9-43de-877f-39e5ee39e894 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:02.622501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6c505-a1d9-43de-877f-39e5ee39e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:02.677770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-56f6c505-a1d9-43de-877f-39e5ee39e894 None None] MFA Rules not processed for user `926d4c4971f1425b98aa5828f802fb21`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:02.679468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-56f6c505-a1d9-43de-877f-39e5ee39e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:02.919384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1089/2183] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:50:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:02.920809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:02.933225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8096605-cb12-4ff7-bb31-c703aebb1247 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:02.940963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:02.941870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:02.942356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:02.942635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:02.943449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:02.945694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:03.216859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1095/2184] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:02 2022] POST /identity/v3/projects => generated 371 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:03.220428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6c505-a1d9-43de-877f-39e5ee39e894 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:03.232357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6c505-a1d9-43de-877f-39e5ee39e894 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:03.302729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:03.303407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:03.303691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:03.303978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:03.304674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:03.306664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:03.614852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1090/2185] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:03 2022] POST /identity/v3/users => generated 292 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:03.618871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:03.630319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c107-d478-40b6-95af-54a4646307e9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:03.638177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:03.638689 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:03.638980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:03.639401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:03.640436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:03.642183 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:03.654204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1096/2186] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:03.659034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:03.670203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364bb6e-3873-4e2f-9b7c-21affac242e4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:03.676335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:03.677295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:03.677575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:03.677848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] PATH_INFO: `/v3/projects/156f859134cf41cf985d337c94c6ed77/users/e8543b358034424b89084f01be07cabe/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:03.694737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156f859134cf41cf985d337c94c6ed77, user_id=e8543b358034424b89084f01be07cabe, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:03.697804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:03.884689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1091/2187] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:03 2022] PUT /identity/v3/projects/156f859134cf41cf985d337c94c6ed77/users/e8543b358034424b89084f01be07cabe/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:03.889217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:03.908608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321df32e-b81c-4d74-a2be-59080f1b55b9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:04.029563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200ab2-46a1-427a-9d12-4b7f0318adca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:04.031148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200ab2-46a1-427a-9d12-4b7f0318adca admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:04.031640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200ab2-46a1-427a-9d12-4b7f0318adca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:04.032131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200ab2-46a1-427a-9d12-4b7f0318adca admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:04.033476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76200ab2-46a1-427a-9d12-4b7f0318adca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:04.036211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76200ab2-46a1-427a-9d12-4b7f0318adca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:04.054974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1097/2188] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:04.059211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:04.088276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:04.097828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c5f12-a897-406a-b8fa-7587e9428bb5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:04.108825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:04.108825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:04.108825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:04.108825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] PATH_INFO: `/v3/projects/156f859134cf41cf985d337c94c6ed77/users/e8543b358034424b89084f01be07cabe/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:04.110961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156f859134cf41cf985d337c94c6ed77, user_id=e8543b358034424b89084f01be07cabe, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:04.114737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:04.159575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1092/2189] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:04 2022] PUT /identity/v3/projects/156f859134cf41cf985d337c94c6ed77/users/e8543b358034424b89084f01be07cabe/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:05.603101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7180c-ab44-4b65-b8dc-89782388bf6f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:05.604554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7180c-ab44-4b65-b8dc-89782388bf6f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:05.605194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7180c-ab44-4b65-b8dc-89782388bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:05.653414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-adb7180c-ab44-4b65-b8dc-89782388bf6f None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:05.655713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-adb7180c-ab44-4b65-b8dc-89782388bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:05.768225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1098/2190] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:50:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:05.781307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:05.783130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:05.887452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7014239-cd95-4cd5-890c-d4cd8a5f11a7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:05.892653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:05.893351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:05.893634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:05.893915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:05.902778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:05.904393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:05.921830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1093/2191] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:50:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:50:14.756229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35babcf3-c66c-40a4-910c-b4d6f55e227c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:14.756816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35babcf3-c66c-40a4-910c-b4d6f55e227c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:14.757266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35babcf3-c66c-40a4-910c-b4d6f55e227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:14.838200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-35babcf3-c66c-40a4-910c-b4d6f55e227c None None] MFA Rules not processed for user `e8543b358034424b89084f01be07cabe`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:14.839455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-35babcf3-c66c-40a4-910c-b4d6f55e227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:14.952680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:15.056522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1099/2192] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:15.072428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:15.075663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:15.267555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839b3cf-f5e5-49fb-ae02-7cdd24fd383b service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:15.275970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c4154-7db7-4a8c-985e-11daf681f6b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:15.276746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c4154-7db7-4a8c-985e-11daf681f6b6 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:15.277144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c4154-7db7-4a8c-985e-11daf681f6b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:15.277499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c4154-7db7-4a8c-985e-11daf681f6b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:15.400868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c4154-7db7-4a8c-985e-11daf681f6b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:15.402877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c4154-7db7-4a8c-985e-11daf681f6b6 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:15.425593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1094/2193] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:19.961522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f11660-289b-4090-ab93-a88bcfb73c50 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:19.962212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f11660-289b-4090-ab93-a88bcfb73c50 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:19.962212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f11660-289b-4090-ab93-a88bcfb73c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.002727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-47f11660-289b-4090-ab93-a88bcfb73c50 None None] MFA Rules not processed for user `926d4c4971f1425b98aa5828f802fb21`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:20.004268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47f11660-289b-4090-ab93-a88bcfb73c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:20.023380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1100/2194] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:50:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:21.839808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a702df4-77d3-4088-8f1a-d392ca652c9f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:21.840491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a702df4-77d3-4088-8f1a-d392ca652c9f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:21.840491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a702df4-77d3-4088-8f1a-d392ca652c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:21.861574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9a702df4-77d3-4088-8f1a-d392ca652c9f None None] MFA Rules not processed for user `eb483b1aee944da8910dd7468a0ef4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:21.862850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a702df4-77d3-4088-8f1a-d392ca652c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:21.878319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:21.949445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1095/2195] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:23.252215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d961b-58df-4901-924c-ca2601759bcd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:23.253141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d961b-58df-4901-924c-ca2601759bcd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:23.253141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d961b-58df-4901-924c-ca2601759bcd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:23.255222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1101/2196] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:50:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:50:23.261449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bc9c8-9792-42a4-84df-3603645308ad None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:23.262772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bc9c8-9792-42a4-84df-3603645308ad None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:23.263266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bc9c8-9792-42a4-84df-3603645308ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:23.290379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-693bc9c8-9792-42a4-84df-3603645308ad None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:23.292319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-693bc9c8-9792-42a4-84df-3603645308ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:23.304690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1096/2197] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:50:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:29.993408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=13 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:31.506845 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d961b-58df-4901-924c-ca2601759bcd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:31.509718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c59d961b-58df-4901-924c-ca2601759bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:31.665615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d961b-58df-4901-924c-ca2601759bcd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:31.669407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac8989-b6c3-433a-8265-68902d170bbb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:31.670770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac8989-b6c3-433a-8265-68902d170bbb service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:31.671252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac8989-b6c3-433a-8265-68902d170bbb service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:31.671538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac8989-b6c3-433a-8265-68902d170bbb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:31.693162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac8989-b6c3-433a-8265-68902d170bbb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:31.693162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac8989-b6c3-433a-8265-68902d170bbb service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:31.703018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1102/2198] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:32.065605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ceb7e-0112-436d-b137-6c985e2b77b1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.066725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ceb7e-0112-436d-b137-6c985e2b77b1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.066725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ceb7e-0112-436d-b137-6c985e2b77b1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.069187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1097/2199] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:50:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:50:32.075143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c33d7d-53d5-4550-888b-c4712a50a616 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.075440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c33d7d-53d5-4550-888b-c4712a50a616 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.076260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c33d7d-53d5-4550-888b-c4712a50a616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.099806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c5c33d7d-53d5-4550-888b-c4712a50a616 None None] MFA Rules not processed for user `a48b0fdacb2c43649916304e1237f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:32.101080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c5c33d7d-53d5-4550-888b-c4712a50a616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:32.176275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1103/2200] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:50:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:32.190265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ceb7e-0112-436d-b137-6c985e2b77b1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:32.203413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ceb7e-0112-436d-b137-6c985e2b77b1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:32.214639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8b262-0bef-4733-a978-7dd672474217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:32.215283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8b262-0bef-4733-a978-7dd672474217 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.215568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8b262-0bef-4733-a978-7dd672474217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.215849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8b262-0bef-4733-a978-7dd672474217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.226923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8b262-0bef-4733-a978-7dd672474217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:32.228314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8b262-0bef-4733-a978-7dd672474217 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:32.237623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1098/2201] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:32.329041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1cc95-ac93-4df9-8078-1e97876af19b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.329334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1cc95-ac93-4df9-8078-1e97876af19b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.329615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1cc95-ac93-4df9-8078-1e97876af19b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.331024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1104/2202] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:50:32.336806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882040-b58c-40a1-96e3-485ca0c3a8e4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.337100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882040-b58c-40a1-96e3-485ca0c3a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.337410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882040-b58c-40a1-96e3-485ca0c3a8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.358604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c3882040-b58c-40a1-96e3-485ca0c3a8e4 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:32.359891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c3882040-b58c-40a1-96e3-485ca0c3a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:32.370504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1099/2203] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:50:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:32.385916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1cc95-ac93-4df9-8078-1e97876af19b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:32.415409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1cc95-ac93-4df9-8078-1e97876af19b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:32.420223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:32.420985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.421273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.421553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.438496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:32.440125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:32.452856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1105/2204] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:32.738214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.741166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.741166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.759411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:32.761147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:32.770792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1100/2205] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 18 10:50:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:32.786461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:32.800947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07902f-9833-4ba3-b777-3cff24c5838f service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:32.809422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a77b6-a9c6-4479-b214-21b4f380ad37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:32.810785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a77b6-a9c6-4479-b214-21b4f380ad37 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:32.811248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a77b6-a9c6-4479-b214-21b4f380ad37 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:32.811591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a77b6-a9c6-4479-b214-21b4f380ad37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:32.823083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a77b6-a9c6-4479-b214-21b4f380ad37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:32.825496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a77b6-a9c6-4479-b214-21b4f380ad37 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:32.837730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1106/2206] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:50:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:50:37.262841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:37.265085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:37.340646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ab4df-a9bf-443a-b07b-dbef506ec163 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:37.346455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:37.347351 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:37.347636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:37.347955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] PATH_INFO: `/v3/users/49e86a3f10b64ef38664013e197782c3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:37.349247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e86a3f10b64ef38664013e197782c3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:37.351207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:37.653028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e86a3f10b64ef38664013e197782c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:37.660888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e86a3f10b64ef38664013e197782c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:37.673541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e86a3f10b64ef38664013e197782c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:37.760718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e86a3f10b64ef38664013e197782c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:37.761375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e86a3f10b64ef38664013e197782c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:37.768999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1101/2207] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:37 2022] DELETE /identity/v3/users/49e86a3f10b64ef38664013e197782c3 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:38.170329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:38.170854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:38.170974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:38.191079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:38.192306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:38.261506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1107/2208] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:50:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:38.274823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:38.276266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:38.289328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:38.312608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:38.312608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:38.361700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:38.382264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94af853-1fc4-455e-92e0-6ca5b7c568ff admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:38.391363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f310a7-58fe-4159-a2b5-6975f265b745 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:38.392099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f310a7-58fe-4159-a2b5-6975f265b745 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:38.401944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f310a7-58fe-4159-a2b5-6975f265b745 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:38.401944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f310a7-58fe-4159-a2b5-6975f265b745 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:38.402136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73ec6-b792-46df-a582-490e01d8d39f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:38.402239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f310a7-58fe-4159-a2b5-6975f265b745 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:38.402618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:38.403499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f310a7-58fe-4159-a2b5-6975f265b745 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:38.403697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:38.404153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:38.404625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] PATH_INFO: `/v3/projects/e8ea247cf5a540768bbbc7260b74f11c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:38.406143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8ea247cf5a540768bbbc7260b74f11c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:38.408105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:38.414014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1102/2209] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:50:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:50:38.602585 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8ea247cf5a540768bbbc7260b74f11c', 'request_id': 'req-2eb896b6-2b77-4e72-8c0c-5680800f3add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:38.689852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8ea247cf5a540768bbbc7260b74f11c', 'request_id': 'req-2eb896b6-2b77-4e72-8c0c-5680800f3add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:38.702396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8ea247cf5a540768bbbc7260b74f11c', 'request_id': 'req-2eb896b6-2b77-4e72-8c0c-5680800f3add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:38.715592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] The token cache is being invalidate because project e8ea247cf5a540768bbbc7260b74f11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:38.716391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:38.717944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eb896b6-2b77-4e72-8c0c-5680800f3add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:38.720355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1108/2210] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:38 2022] DELETE /identity/v3/projects/e8ea247cf5a540768bbbc7260b74f11c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:38.726494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef3c2-f5fe-4955-b68a-7adefdb57262 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:38.726841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef3c2-f5fe-4955-b68a-7adefdb57262 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:38.727159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef3c2-f5fe-4955-b68a-7adefdb57262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:38.752170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-da4ef3c2-f5fe-4955-b68a-7adefdb57262 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:38.753488 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-da4ef3c2-f5fe-4955-b68a-7adefdb57262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:38.841285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1103/2211] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:38.850115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa80e0-e2c7-4e40-8a96-6d1a835970ba None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:38.850654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa80e0-e2c7-4e40-8a96-6d1a835970ba None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:38.850763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa80e0-e2c7-4e40-8a96-6d1a835970ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:38.895829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7bfa80e0-e2c7-4e40-8a96-6d1a835970ba None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:38.899109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7bfa80e0-e2c7-4e40-8a96-6d1a835970ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:38.917841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1109/2212] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:38.926858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef3c2-f5fe-4955-b68a-7adefdb57262 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:38.953266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef3c2-f5fe-4955-b68a-7adefdb57262 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:38.961960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:38.963510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:38.964432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:38.964704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:38.966928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:38.967256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:38.972381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:39.000311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1104/2213] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:39.001568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa80e0-e2c7-4e40-8a96-6d1a835970ba None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:39.020665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa80e0-e2c7-4e40-8a96-6d1a835970ba None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:39.041239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:39.044107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:39.044827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:39.045711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:39.046613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:39.056526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:39.252262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1110/2214] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:39 2022] POST /identity/v3/projects => generated 363 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:39.253258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:39.267989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c81c1-b3d7-42b9-8213-53b98aa8deb6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:39.359100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:39.359947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:39.360241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:39.360518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:39.376302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:39.378339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:39.584063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1105/2215] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:39 2022] POST /identity/v3/users => generated 282 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:39.584850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:39.609353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fc99e-9c1d-43f2-a13f-e67706ed5640 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:39.609353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:39.609353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:39.609353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:39.609838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:39.609838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:39.609838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:39.621222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1111/2216] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:39.627835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:39.643284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeabe64-7b7d-4107-8dfa-e0c0d0c068d8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:39.654814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84910630-f89c-4be3-a9a6-93835fbd96fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:39.655531 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910630-f89c-4be3-a9a6-93835fbd96fe admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:39.655822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910630-f89c-4be3-a9a6-93835fbd96fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:39.656117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910630-f89c-4be3-a9a6-93835fbd96fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:39.656915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84910630-f89c-4be3-a9a6-93835fbd96fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:39.658603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84910630-f89c-4be3-a9a6-93835fbd96fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:39.673347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1106/2217] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:39.677454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:39.692544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39c61d-fc79-459e-bba0-b9bd489e0de1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:39.704536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:39.705527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:39.705828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:39.706104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] PATH_INFO: `/v3/projects/84a371f132714a76858fac5d2e0c4aeb/users/8e2d22dabcc2425ba2c8d87391da5b97/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:39.723324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a371f132714a76858fac5d2e0c4aeb, user_id=8e2d22dabcc2425ba2c8d87391da5b97, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:39.726300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:39.844095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1112/2218] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:39 2022] PUT /identity/v3/projects/84a371f132714a76858fac5d2e0c4aeb/users/8e2d22dabcc2425ba2c8d87391da5b97/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:39.846862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:39.847379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:39.847924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:39.861023 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:39.886081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] MFA Rules not processed for user `8e2d22dabcc2425ba2c8d87391da5b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:39.887461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:40.000368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1107/2219] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:50:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:40.014354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:40.016720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:40.132848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322a86e-6835-477c-b61f-efe6837dd441 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:40.140533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4384c-fb4a-45ea-983d-42045353e150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:40.141795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4384c-fb4a-45ea-983d-42045353e150 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:40.144647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4384c-fb4a-45ea-983d-42045353e150 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:40.145379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4384c-fb4a-45ea-983d-42045353e150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:40.160251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4384c-fb4a-45ea-983d-42045353e150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:40.161788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4384c-fb4a-45ea-983d-42045353e150 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:40.167468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:40.172872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1113/2220] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:40 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:49.995764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=30 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:53.529146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:53.530976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:53.607256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f824a76-aa5d-4756-b08d-085b911c892b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:53.616005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:53.616795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:53.617090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:53.617373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:53.619266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:53.724258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:53.725913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:53.741759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1108/2221] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:02.386981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047ae9-2ffd-4971-b432-736ec78f78bc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:02.389835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047ae9-2ffd-4971-b432-736ec78f78bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:02.389835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047ae9-2ffd-4971-b432-736ec78f78bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:02.406080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-80047ae9-2ffd-4971-b432-736ec78f78bc None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:02.407452 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-80047ae9-2ffd-4971-b432-736ec78f78bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:02.483617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1114/2222] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 18 10:51:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:02.508936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:02.510442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:02.596863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75483aad-b704-4627-a7f7-446b89d5b504 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:02.602039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:02.602706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:02.602996 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:02.603275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:02.613137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:02.615436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:02.623882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1109/2223] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:51:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:51:12.474644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=25 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:12.619941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=82 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:18.830660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870892f-ccf1-46c7-8620-7347b401ff23 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:18.831171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870892f-ccf1-46c7-8620-7347b401ff23 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:18.831171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870892f-ccf1-46c7-8620-7347b401ff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:18.850172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1870892f-ccf1-46c7-8620-7347b401ff23 None None] MFA Rules not processed for user `3954fd4abf6b4c3a82280c762292a3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:18.851420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1870892f-ccf1-46c7-8620-7347b401ff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:18.920371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1115/2224] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:20.182036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:20.184148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:20.196501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f17f-3103-492b-8510-4f038a692484 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:20.201119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dc6e3-52df-40f2-b009-12d09492f7c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:20.201370 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dc6e3-52df-40f2-b009-12d09492f7c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.201556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dc6e3-52df-40f2-b009-12d09492f7c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.202006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dc6e3-52df-40f2-b009-12d09492f7c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.203855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cf0dc6e3-52df-40f2-b009-12d09492f7c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:20.271349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dc6e3-52df-40f2-b009-12d09492f7c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:20.272827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dc6e3-52df-40f2-b009-12d09492f7c5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:20.283526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1110/2225] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:27.433224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870892f-ccf1-46c7-8620-7347b401ff23 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:27.515960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870892f-ccf1-46c7-8620-7347b401ff23 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:27.521150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:27.522083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:27.522366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:27.522644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] PATH_INFO: `/v3/users/8e2d22dabcc2425ba2c8d87391da5b97` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:27.523865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e2d22dabcc2425ba2c8d87391da5b97)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:27.523865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:27.728529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e2d22dabcc2425ba2c8d87391da5b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:27.741521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e2d22dabcc2425ba2c8d87391da5b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:27.857086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e2d22dabcc2425ba2c8d87391da5b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:27.857697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e2d22dabcc2425ba2c8d87391da5b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:27.858322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e2d22dabcc2425ba2c8d87391da5b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:27.872494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1116/2226] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:27 2022] DELETE /identity/v3/users/8e2d22dabcc2425ba2c8d87391da5b97 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:27.877782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:27.878313 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:27.878474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:27.896431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:27.897700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:27.960374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1111/2227] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:27.971982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:27.973436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:28.005238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:28.045714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a20225-ccac-4580-a167-2fddcd2c5ada admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:28.050636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa60c2-38a2-40d4-b171-03b4761b6a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:28.051332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa60c2-38a2-40d4-b171-03b4761b6a70 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:28.051619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa60c2-38a2-40d4-b171-03b4761b6a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:28.051926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa60c2-38a2-40d4-b171-03b4761b6a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:28.061915 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa60c2-38a2-40d4-b171-03b4761b6a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:28.063423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa60c2-38a2-40d4-b171-03b4761b6a70 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:28.074927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1117/2228] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:28.710457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:28.715569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:28.731885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cdd5d-f74e-44fa-a3bb-b2909a8691fd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:28.738582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:28.739763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:28.739936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:28.740224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] PATH_INFO: `/v3/projects/84a371f132714a76858fac5d2e0c4aeb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:28.741497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a371f132714a76858fac5d2e0c4aeb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:28.743557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:28.905044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a371f132714a76858fac5d2e0c4aeb', 'request_id': 'req-8701ce78-ca5c-47ba-9ce1-547413926b1e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:28.911461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a371f132714a76858fac5d2e0c4aeb', 'request_id': 'req-8701ce78-ca5c-47ba-9ce1-547413926b1e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.039341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a371f132714a76858fac5d2e0c4aeb', 'request_id': 'req-8701ce78-ca5c-47ba-9ce1-547413926b1e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.050634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] The token cache is being invalidate because project 84a371f132714a76858fac5d2e0c4aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:29.050928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.051567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8701ce78-ca5c-47ba-9ce1-547413926b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.053169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1112/2229] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:28 2022] DELETE /identity/v3/projects/84a371f132714a76858fac5d2e0c4aeb => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:29.058764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e825-c441-4c40-9de1-bc80444a5953 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.059312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e825-c441-4c40-9de1-bc80444a5953 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.059312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e825-c441-4c40-9de1-bc80444a5953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.078266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4bd5e825-c441-4c40-9de1-bc80444a5953 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:29.079479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4bd5e825-c441-4c40-9de1-bc80444a5953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:29.142491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1118/2230] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:29.151712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419fc8b-cbe0-4b4e-b22e-cb066c47d530 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.152210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419fc8b-cbe0-4b4e-b22e-cb066c47d530 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.152576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419fc8b-cbe0-4b4e-b22e-cb066c47d530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.171963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0419fc8b-cbe0-4b4e-b22e-cb066c47d530 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:29.173156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0419fc8b-cbe0-4b4e-b22e-cb066c47d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:29.182282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1113/2231] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:29.187715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e825-c441-4c40-9de1-bc80444a5953 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.197793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e825-c441-4c40-9de1-bc80444a5953 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.202744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.203415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.203703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.203982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.204722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:29.205064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.207022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.219738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1119/2232] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:51:29.224551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419fc8b-cbe0-4b4e-b22e-cb066c47d530 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.235104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419fc8b-cbe0-4b4e-b22e-cb066c47d530 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.240616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.241396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.241671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.241949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.242679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.244731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.255092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:29.364812 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1114/2233] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:29 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:29.366858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.378130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b073436-cf08-460e-9b21-1171d600163b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.451879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.455227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.455227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.455227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.455227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.456000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.600770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1120/2234] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:29 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:29.605284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.617192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3253f53-1245-45c6-bb90-5c5683a12374 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.622219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.622970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.623337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.623705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.624444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.626174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.637606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1115/2235] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:51:29.641878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.652770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e839e-63fb-4c2a-ab4e-4b4a8e076740 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.657769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39759a-124c-4f09-8674-cdd103d0b7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.661935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39759a-124c-4f09-8674-cdd103d0b7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.661935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39759a-124c-4f09-8674-cdd103d0b7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.661935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39759a-124c-4f09-8674-cdd103d0b7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.661935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39759a-124c-4f09-8674-cdd103d0b7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.661935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39759a-124c-4f09-8674-cdd103d0b7ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.674425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1121/2236] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:29.678905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.689439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26881df5-9647-4df5-959f-ba7402439e51 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.694110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.695061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.695342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.695622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] PATH_INFO: `/v3/projects/e6516b4ff3d14c56972ec151cf040780/users/abc3e0a80f734ac68e25835669acc522/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.711302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6516b4ff3d14c56972ec151cf040780, user_id=abc3e0a80f734ac68e25835669acc522, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.714609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.842246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1116/2237] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:29 2022] PUT /identity/v3/projects/e6516b4ff3d14c56972ec151cf040780/users/abc3e0a80f734ac68e25835669acc522/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:29.853407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54c5d1-5f51-4eae-b762-d5b3f59bfee9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.853652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54c5d1-5f51-4eae-b762-d5b3f59bfee9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.853872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54c5d1-5f51-4eae-b762-d5b3f59bfee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.883552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fd54c5d1-5f51-4eae-b762-d5b3f59bfee9 None None] MFA Rules not processed for user `abc3e0a80f734ac68e25835669acc522`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:29.884835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd54c5d1-5f51-4eae-b762-d5b3f59bfee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:29.913878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:29.969729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1122/2238] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:29.982058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.984090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:30.056420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bdf7b-6b7b-43d0-bbd9-75790d1149cb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:30.061331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31cb69-8bb0-446b-9daa-7258ba0b1563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:30.062015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31cb69-8bb0-446b-9daa-7258ba0b1563 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:30.062310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31cb69-8bb0-446b-9daa-7258ba0b1563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:30.062581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31cb69-8bb0-446b-9daa-7258ba0b1563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:30.073569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31cb69-8bb0-446b-9daa-7258ba0b1563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:30.075180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31cb69-8bb0-446b-9daa-7258ba0b1563 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:30.085576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1117/2239] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:30.117065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54c5d1-5f51-4eae-b762-d5b3f59bfee9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:30.194701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54c5d1-5f51-4eae-b762-d5b3f59bfee9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:30.200569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:30.201491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:30.201783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:30.202065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] PATH_INFO: `/v3/users/abc3e0a80f734ac68e25835669acc522` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:30.203437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc3e0a80f734ac68e25835669acc522)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:30.205561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:30.525942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc3e0a80f734ac68e25835669acc522'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:30.540426 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc3e0a80f734ac68e25835669acc522'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:30.616927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc3e0a80f734ac68e25835669acc522'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:30.617609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc3e0a80f734ac68e25835669acc522'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:30.618227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc3e0a80f734ac68e25835669acc522'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:30.632847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1123/2240] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:30 2022] DELETE /identity/v3/users/abc3e0a80f734ac68e25835669acc522 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:30.637713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:30.638010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:30.638295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:30.657832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:30.659175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:30.724026 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1118/2241] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:30.736341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:30.737780 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:30.790786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:30.804355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6ee3a-e831-4dec-9753-fbaa446691c9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:30.812310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735829a-3b47-4223-a3c4-b42dd33d81e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:30.813271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735829a-3b47-4223-a3c4-b42dd33d81e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:30.813793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735829a-3b47-4223-a3c4-b42dd33d81e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:30.814358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735829a-3b47-4223-a3c4-b42dd33d81e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:30.829613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735829a-3b47-4223-a3c4-b42dd33d81e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:30.831241 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735829a-3b47-4223-a3c4-b42dd33d81e3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:30.842209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1124/2242] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:51:31.412741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:31.414253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:31.427728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c287-92f5-4547-9d51-7c98e735c0ea None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:31.433499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:31.434496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.434740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.435014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] PATH_INFO: `/v3/projects/e6516b4ff3d14c56972ec151cf040780` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.436334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6516b4ff3d14c56972ec151cf040780)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:31.438295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.630395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6516b4ff3d14c56972ec151cf040780', 'request_id': 'req-f743e128-f572-4766-9f1c-8405227bd213'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.632659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:31.638258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6516b4ff3d14c56972ec151cf040780', 'request_id': 'req-f743e128-f572-4766-9f1c-8405227bd213'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.753241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6516b4ff3d14c56972ec151cf040780', 'request_id': 'req-f743e128-f572-4766-9f1c-8405227bd213'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.764964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] The token cache is being invalidate because project e6516b4ff3d14c56972ec151cf040780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:31.765247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.765894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f743e128-f572-4766-9f1c-8405227bd213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.767440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1119/2243] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:31 2022] DELETE /identity/v3/projects/e6516b4ff3d14c56972ec151cf040780 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:38.611112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059204-dd62-4cc0-9610-0849873e0e77 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:38.611674 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059204-dd62-4cc0-9610-0849873e0e77 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:38.611797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059204-dd62-4cc0-9610-0849873e0e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:38.631858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f3059204-dd62-4cc0-9610-0849873e0e77 None None] MFA Rules not processed for user `e7a738986f87470681c6a578599a089f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:38.633186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f3059204-dd62-4cc0-9610-0849873e0e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:38.703242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1125/2244] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:43.045006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:48.694792 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:56.926807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.927374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.927374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.949402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] MFA Rules not processed for user `3dbb489073d94425ad1a1b350768c428`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:56.950623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:57.010515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1120/2245] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:06.017981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.018487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.018587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.038059 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] MFA Rules not processed for user `7e032c8e5b024fa7a41cd1090e0f1ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:06.039244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:06.100729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1126/2246] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:07.003232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=25 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:16.093911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=25 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:20.123398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:20.124729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:20.185760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1248a-7afb-4da5-8e33-ce2c1c0c1488 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:20.189349 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:20.190002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:20.190289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:20.190568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:20.192346 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:20.265182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:20.266654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:20.278869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1121/2247] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:23.286559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:23.288235 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:23.355918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9591d-25d8-4a22-97d6-ce959f060ae2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:23.359635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:23.360300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:23.360724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:23.361057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:23.362988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:23.435758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:23.437538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:23.452696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1127/2248] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:24.606142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:24.615624 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147dfd-f39c-461a-b549-8d38a8efa668 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:24.620105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30109d41-cb98-4001-a2ab-264a60a98a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:24.620749 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30109d41-cb98-4001-a2ab-264a60a98a42 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:24.621122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30109d41-cb98-4001-a2ab-264a60a98a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:24.621402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30109d41-cb98-4001-a2ab-264a60a98a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:24.623238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30109d41-cb98-4001-a2ab-264a60a98a42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:24.637402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30109d41-cb98-4001-a2ab-264a60a98a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:24.638847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30109d41-cb98-4001-a2ab-264a60a98a42 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:24.651055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1122/2249] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:25.598006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:25.607989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb12263-401e-46c3-a545-6f376c58a4e1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:25.611714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:25.612359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:25.612646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:25.612952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:25.614760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:25.627424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:25.631347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:25.639329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1128/2250] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:34.645156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=60 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:35.634281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=54 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:50.234696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:50.235354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:50.235672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:50.283379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] MFA Rules not processed for user `3954fd4abf6b4c3a82280c762292a3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:50.285282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:50.321183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1123/2251] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:50.684042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:50.737039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335a958-f7e6-4af3-8deb-d5f41afa4d50 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:50.741382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:50.742089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:50.742369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:50.742647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:50.744476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:50.769846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:50.771460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:50.782553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1129/2252] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:51.300450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:51.301892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:51.402815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b16ba-c258-4153-b123-26d6924bb940 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:51.407914 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:51.408726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:51.409001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:51.409278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:51.411090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:51.424316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:51.425954 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:51.436344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1124/2253] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:52.835511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:52.837609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:52.855619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39e200-6c11-4eb5-b371-bc8d87205362 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:52.861392 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:52.862071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:52.862354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:52.862632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:52.864543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:52.879774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:52.881565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:52.893591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1130/2254] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:52:57.598743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:57.611704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c0200-1bb7-4a0e-9196-2086f4bc205a service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:57.618172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:57.618867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:57.619156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:57.619436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:57.621298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:57.632330 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:57.633967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:57.643316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1125/2255] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:52:58.106378 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:58.108382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:58.121036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f5e2e-861f-497e-b19b-b03c01047ac0 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:58.130074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:58.130074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:58.130074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:58.130074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:58.130447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:58.140978 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:58.142591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:58.152609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1131/2256] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:59.135718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:59.154160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901d9d-9fe1-45b0-96d1-7f45be6f1e1c service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:59.174930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:59.175704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:59.176055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:59.176433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:59.178597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:59.196354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:59.196592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:59.221453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1126/2257] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:01.335611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:01.345682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cbfdc-8ac2-4460-892a-cf143ec15c32 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:01.349465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:01.350116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:01.350396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:01.350743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:01.352897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:01.454710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:01.456337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:01.482808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1132/2258] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:04.767737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:04.769685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:04.785001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318021b6-cd39-496c-8fb4-5395cc977cfb service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:04.791591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:04.792303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:04.792587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:04.800734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:04.800734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:04.809823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:04.811577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:04.824369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1127/2259] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:06.075179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:06.078419 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:06.106487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0526e-ed34-4213-bf04-9f76ff0e498a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:06.112697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:06.113745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:06.114035 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:06.114311 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] PATH_INFO: `/v3/users/e7a738986f87470681c6a578599a089f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:06.116136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a738986f87470681c6a578599a089f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:06.118293 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:06.169968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:06.183186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fd7cb-9cd2-43e0-8508-effaa7d67597 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:06.189019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:06.189744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:06.190136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:06.190433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:06.192476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:06.204696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:06.206518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:06.217416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1128/2260] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:53:06.428862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:06.436097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a738986f87470681c6a578599a089f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:06.454733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a738986f87470681c6a578599a089f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:06.580033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a738986f87470681c6a578599a089f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:06.580672 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a738986f87470681c6a578599a089f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:06.581346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c80cc4a2-8b34-47ef-a2f3-aa2c78476c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a738986f87470681c6a578599a089f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:06.596828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1133/2261] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:06 2022] DELETE /identity/v3/users/e7a738986f87470681c6a578599a089f => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:07.063629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:07.082929 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3ec0a-a00a-424f-bac5-c8d525b6564a service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:07.208543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:07.230260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:07.231421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:07.231758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:07.232111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] PATH_INFO: `/v3/projects/7184d0f3256344a2af18c639787e538f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:07.235867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7184d0f3256344a2af18c639787e538f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:07.237169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:07.434371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7184d0f3256344a2af18c639787e538f', 'request_id': 'req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:07.441494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7184d0f3256344a2af18c639787e538f', 'request_id': 'req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:07.562439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7184d0f3256344a2af18c639787e538f', 'request_id': 'req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:07.575609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] The token cache is being invalidate because project 7184d0f3256344a2af18c639787e538f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:07.575994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:07.576728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0dc45258-e7a9-4bd9-8e7a-906dd6b748f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:07.578429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1129/2262] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:07 2022] DELETE /identity/v3/projects/7184d0f3256344a2af18c639787e538f => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:07.584888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247f28-f6d3-4143-baa0-c7fccaba7c6a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:07.589864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247f28-f6d3-4143-baa0-c7fccaba7c6a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:07.589864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247f28-f6d3-4143-baa0-c7fccaba7c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:07.654618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-25247f28-f6d3-4143-baa0-c7fccaba7c6a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:07.655872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-25247f28-f6d3-4143-baa0-c7fccaba7c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:07.720711 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1134/2263] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:07.730148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b0a24-2dc5-481c-9946-b78c23ecaeb6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:07.730982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b0a24-2dc5-481c-9946-b78c23ecaeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:07.731861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b0a24-2dc5-481c-9946-b78c23ecaeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:07.751114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b64b0a24-2dc5-481c-9946-b78c23ecaeb6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:07.752326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b64b0a24-2dc5-481c-9946-b78c23ecaeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:07.761483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1130/2264] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:07.765914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247f28-f6d3-4143-baa0-c7fccaba7c6a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:07.783942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247f28-f6d3-4143-baa0-c7fccaba7c6a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:07.789960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:07.790750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:07.791098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:07.791450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:07.792257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:07.792634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:07.794372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:07.818242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1135/2265] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:53:07.822741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b0a24-2dc5-481c-9946-b78c23ecaeb6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:07.833229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b0a24-2dc5-481c-9946-b78c23ecaeb6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:07.839050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:07.839609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:07.839900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:07.840183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:07.841262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:07.843214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:08.032549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1131/2266] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:07 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:08.036720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:08.056556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c129ce-abf1-4c8b-acbd-ffbdc1d74937 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:08.143458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:08.144537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:08.145109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:08.145633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:08.146760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:08.150831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:08.358933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1136/2267] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:08 2022] POST /identity/v3/users => generated 288 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:08.363275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:08.374946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b3af2-8869-4e80-b1ae-4d66bc1cec6e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:08.380275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:08.381018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:08.381307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:08.381675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:08.382620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:08.384288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:08.395705 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1132/2268] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:08.405560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:08.417139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018347f-6d14-4898-904b-a31536f8b090 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:08.421921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466919db-fcf1-4608-b241-d07af993d68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:08.422630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466919db-fcf1-4608-b241-d07af993d68a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:08.422873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466919db-fcf1-4608-b241-d07af993d68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:08.423143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466919db-fcf1-4608-b241-d07af993d68a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:08.423865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466919db-fcf1-4608-b241-d07af993d68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:08.425504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466919db-fcf1-4608-b241-d07af993d68a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:08.427003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:08.438767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1137/2269] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:08.442751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:08.453919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36758c-8029-4bd8-b4c6-3861029dc9a2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:08.458889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:08.460554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:08.461149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:08.461508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] PATH_INFO: `/v3/projects/0d28a0aa7f02406f8af879023498292d/users/38403b569e9f41c7ba4864388b16e1c4/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:08.490615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d28a0aa7f02406f8af879023498292d, user_id=38403b569e9f41c7ba4864388b16e1c4, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:08.494340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:08.619166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1133/2270] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:08 2022] PUT /identity/v3/projects/0d28a0aa7f02406f8af879023498292d/users/38403b569e9f41c7ba4864388b16e1c4/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:08.630332 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7b09-7a68-43ca-b2ab-1a8d8f78691c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:08.632362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7b09-7a68-43ca-b2ab-1a8d8f78691c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:08.632362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7b09-7a68-43ca-b2ab-1a8d8f78691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:08.661204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-381a7b09-7a68-43ca-b2ab-1a8d8f78691c None None] MFA Rules not processed for user `38403b569e9f41c7ba4864388b16e1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:08.662266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-381a7b09-7a68-43ca-b2ab-1a8d8f78691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:08.760060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1138/2271] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:08.769038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:08.770907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:08.890571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e6fc-0091-4e45-a2ff-7fae0182cd6f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:08.896601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6951400-843d-49df-ae94-c4247054e582 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:08.897391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6951400-843d-49df-ae94-c4247054e582 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:08.897746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6951400-843d-49df-ae94-c4247054e582 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:08.898078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6951400-843d-49df-ae94-c4247054e582 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:08.906732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6951400-843d-49df-ae94-c4247054e582 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:08.908418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6951400-843d-49df-ae94-c4247054e582 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:08.915071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:08.920842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1134/2272] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:53:09.289820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7b09-7a68-43ca-b2ab-1a8d8f78691c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:09.402306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7b09-7a68-43ca-b2ab-1a8d8f78691c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:09.410651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:09.413685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:09.413685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:09.415017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] PATH_INFO: `/v3/users/38403b569e9f41c7ba4864388b16e1c4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:09.417432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38403b569e9f41c7ba4864388b16e1c4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:09.420908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:09.758587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38403b569e9f41c7ba4864388b16e1c4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:09.776159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38403b569e9f41c7ba4864388b16e1c4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:09.861356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38403b569e9f41c7ba4864388b16e1c4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:09.861740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38403b569e9f41c7ba4864388b16e1c4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:09.862847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38403b569e9f41c7ba4864388b16e1c4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:09.888278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1139/2273] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:09 2022] DELETE /identity/v3/users/38403b569e9f41c7ba4864388b16e1c4 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:09.896171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2a072-99a0-422d-9879-70941f5d1085 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:09.896611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2a072-99a0-422d-9879-70941f5d1085 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:09.898405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2a072-99a0-422d-9879-70941f5d1085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:09.921410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8da2a072-99a0-422d-9879-70941f5d1085 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:09.924081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8da2a072-99a0-422d-9879-70941f5d1085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:10.040347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1135/2274] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:10.054808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:10.057112 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:10.163658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25679d-8c1d-4f8c-b02f-44520ec73f62 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:10.167774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:10.169501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:10.169908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:10.170373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:10.181994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:10.183635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:10.207187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1140/2275] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:10.246124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:10.246436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:10.246601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:10.269917 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] MFA Rules not processed for user `eb483b1aee944da8910dd7468a0ef4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:10.271136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:10.338585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1136/2276] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:10.827099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:10.841568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8700a1d-2c47-4ee5-bedb-a2122b2845c3 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:10.853499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf0952-afed-434e-9fb9-68631567d0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:10.854437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf0952-afed-434e-9fb9-68631567d0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:10.854869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf0952-afed-434e-9fb9-68631567d0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:10.855418 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf0952-afed-434e-9fb9-68631567d0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:10.860327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-35bf0952-afed-434e-9fb9-68631567d0e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:10.873807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf0952-afed-434e-9fb9-68631567d0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:10.875890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf0952-afed-434e-9fb9-68631567d0e7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:10.888233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1141/2277] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:11.008099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:11.008099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:11.025098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b419be-53b3-4a4b-a740-a9006bd69664 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:11.047121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:11.048051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.048477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.048764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] PATH_INFO: `/v3/projects/0d28a0aa7f02406f8af879023498292d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.051721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d28a0aa7f02406f8af879023498292d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:11.054527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:11.195092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:11.207433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d28a0aa7f02406f8af879023498292d', 'request_id': 'req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:11.207433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d28a0aa7f02406f8af879023498292d', 'request_id': 'req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:11.289134 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d28a0aa7f02406f8af879023498292d', 'request_id': 'req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:11.303332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] The token cache is being invalidate because project 0d28a0aa7f02406f8af879023498292d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:11.304214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:11.304976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3c7e2604-e043-44ee-ae35-5fcc47aac3a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:11.306792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1137/2278] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:10 2022] DELETE /identity/v3/projects/0d28a0aa7f02406f8af879023498292d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:11.313052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe8a2-3571-4972-b4fb-4230c884b0ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.313700 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe8a2-3571-4972-b4fb-4230c884b0ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.314184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe8a2-3571-4972-b4fb-4230c884b0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.338709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-23bfe8a2-3571-4972-b4fb-4230c884b0ab None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:11.340080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-23bfe8a2-3571-4972-b4fb-4230c884b0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:11.417890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1142/2279] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:11.426484 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258adb-f827-4f00-8d65-752779f49a1c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.426864 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258adb-f827-4f00-8d65-752779f49a1c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.427354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258adb-f827-4f00-8d65-752779f49a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.449622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-58258adb-f827-4f00-8d65-752779f49a1c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:11.450885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-58258adb-f827-4f00-8d65-752779f49a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:11.459326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1138/2280] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:11.473229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfe8a2-3571-4972-b4fb-4230c884b0ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:11.484205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfe8a2-3571-4972-b4fb-4230c884b0ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:11.489120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:11.489826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.490168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.490387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.491747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:11.492055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:11.493718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:11.508479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1143/2281] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:53:11.514379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58258adb-f827-4f00-8d65-752779f49a1c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:11.525094 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58258adb-f827-4f00-8d65-752779f49a1c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:11.529726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:11.530393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.530678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.530953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.531652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:11.533588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:11.667078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1139/2282] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:11 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:11.671644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:11.682606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c509237-a534-46c4-a73d-066c64083edf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:11.749944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:11.750620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.750900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.751174 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.751868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:11.753858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:11.900888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1144/2283] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:11 2022] POST /identity/v3/users => generated 282 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:11.905100 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:11.918015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2915557-4056-4d2d-b944-9921694c16dd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:11.923238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:11.924261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.924261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.924482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.925213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:11.926821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:11.938964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1140/2284] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:11.943276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:11.953958 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700a2db-e626-46dc-a913-917048ec9efe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:11.965205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732a423-3506-4649-9dbd-77c4aca5885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:11.965205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732a423-3506-4649-9dbd-77c4aca5885f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.965205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732a423-3506-4649-9dbd-77c4aca5885f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.965205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732a423-3506-4649-9dbd-77c4aca5885f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:11.965956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732a423-3506-4649-9dbd-77c4aca5885f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:11.965956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732a423-3506-4649-9dbd-77c4aca5885f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:11.973234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1145/2285] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:11.978918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:11.989587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17450ce2-9ba1-4aee-90c5-73cf65323752 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:11.995194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:11.996105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:11.997265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:11.997265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] PATH_INFO: `/v3/projects/f2229dade5d84655b93f1089dc039cef/users/dd3f0800ce9e4f87846c968d5965a167/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:12.013776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2229dade5d84655b93f1089dc039cef, user_id=dd3f0800ce9e4f87846c968d5965a167, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:12.018500 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:12.212175 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1141/2286] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:11 2022] PUT /identity/v3/projects/f2229dade5d84655b93f1089dc039cef/users/dd3f0800ce9e4f87846c968d5965a167/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:12.223961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f095620-0111-4399-8c77-a7161dfc5ca4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:12.224379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f095620-0111-4399-8c77-a7161dfc5ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:12.224737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f095620-0111-4399-8c77-a7161dfc5ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:12.268042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0f095620-0111-4399-8c77-a7161dfc5ca4 None None] MFA Rules not processed for user `dd3f0800ce9e4f87846c968d5965a167`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:12.269305 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f095620-0111-4399-8c77-a7161dfc5ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:12.344927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1146/2287] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:12.348400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:12.387780 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:12.416156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1ee5-a13c-4db8-ab30-858f57d3f767 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:12.421494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:12.422162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:12.422451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:12.422773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:12.423505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:12.425467 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:12.584303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1142/2288] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:12 2022] POST /identity/v3/projects => generated 365 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:12.584430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f095620-0111-4399-8c77-a7161dfc5ca4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:12.595449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f095620-0111-4399-8c77-a7161dfc5ca4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:12.653409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:12.654085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:12.654369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:12.654647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:12.655345 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:12.657349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:12.744001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1147/2289] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:12 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:12.745355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:12.755738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e691b8b-4300-42a0-b70a-091f2e51d8e7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:12.760414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:12.761153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:12.761442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:12.761724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:12.762453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:12.764044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:12.774813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1143/2290] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:12.779013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:12.789398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868bc86-b53a-4bfd-838e-d97a3e5ef8c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:12.793862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:12.794746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:12.795111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:12.795654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] PATH_INFO: `/v3/projects/8688bfab37c44c2096dbc3616418bd98/users/710c72b931b8422abec9c87ea7ba6feb/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:12.812803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8688bfab37c44c2096dbc3616418bd98, user_id=710c72b931b8422abec9c87ea7ba6feb, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:12.815741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:12.923495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1148/2291] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:12 2022] PUT /identity/v3/projects/8688bfab37c44c2096dbc3616418bd98/users/710c72b931b8422abec9c87ea7ba6feb/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:12.928047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:12.939123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff1d7f-ea1d-486c-baae-28272db2e493 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:13.003601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d2dd7-2f6d-4255-a317-038f056083c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:13.003601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d2dd7-2f6d-4255-a317-038f056083c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:13.003601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d2dd7-2f6d-4255-a317-038f056083c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:13.003601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d2dd7-2f6d-4255-a317-038f056083c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:13.004660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d2dd7-2f6d-4255-a317-038f056083c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:13.004937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d2dd7-2f6d-4255-a317-038f056083c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:13.016280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1144/2292] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:13.020557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:13.032983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4c1c-8e5c-4cd7-a998-36d66e459f65 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:13.038738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:13.039763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:13.040053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:13.040353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] PATH_INFO: `/v3/projects/8688bfab37c44c2096dbc3616418bd98/users/710c72b931b8422abec9c87ea7ba6feb/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:13.042629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8688bfab37c44c2096dbc3616418bd98, user_id=710c72b931b8422abec9c87ea7ba6feb, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:13.045377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:13.057248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1149/2293] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:13 2022] PUT /identity/v3/projects/8688bfab37c44c2096dbc3616418bd98/users/710c72b931b8422abec9c87ea7ba6feb/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:13.067713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c362f1-e837-46ec-907f-72befeb77ff1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:13.072283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c362f1-e837-46ec-907f-72befeb77ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:13.072283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c362f1-e837-46ec-907f-72befeb77ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:13.098740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-74c362f1-e837-46ec-907f-72befeb77ff1 None None] MFA Rules not processed for user `710c72b931b8422abec9c87ea7ba6feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:13.099986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-74c362f1-e837-46ec-907f-72befeb77ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:13.180272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1145/2294] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:13.191609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:13.202893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:13.291199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51830e9-dd73-4033-b052-ddcf4229047d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:13.295124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1c73b-86a5-4789-91ec-7333b3d23812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:13.295775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c73b-86a5-4789-91ec-7333b3d23812 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:13.296085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c73b-86a5-4789-91ec-7333b3d23812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:13.296359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c73b-86a5-4789-91ec-7333b3d23812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:13.306091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1c73b-86a5-4789-91ec-7333b3d23812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:13.307539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1c73b-86a5-4789-91ec-7333b3d23812 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:13.320671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1150/2295] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:53:14.917774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c362f1-e837-46ec-907f-72befeb77ff1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:14.985535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c362f1-e837-46ec-907f-72befeb77ff1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:14.990269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:14.991530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:14.991896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:14.992250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] PATH_INFO: `/v3/users/dd3f0800ce9e4f87846c968d5965a167` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:14.993580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3f0800ce9e4f87846c968d5965a167)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:14.995610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:15.329915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3f0800ce9e4f87846c968d5965a167'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:15.338714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3f0800ce9e4f87846c968d5965a167'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:15.352403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3f0800ce9e4f87846c968d5965a167'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:15.488599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3f0800ce9e4f87846c968d5965a167'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:15.489598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3f0800ce9e4f87846c968d5965a167'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:15.501429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1146/2296] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:14 2022] DELETE /identity/v3/users/dd3f0800ce9e4f87846c968d5965a167 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:15.502810 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:15.503284 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:15.504548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:15.522203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:15.523438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:15.588322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1151/2297] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:15.598711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:15.600694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:15.664060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274463a-59d3-4e48-ac29-020874594ea0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:15.668964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:15.669734 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:15.670106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:15.670487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:15.682092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:15.683560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:15.696087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1147/2298] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:15.880535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:16.426893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:16.429608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:16.441724 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3581d0-efff-4276-b74d-1e10d2499c0a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:16.447411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:16.448624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:16.448759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:16.449207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] PATH_INFO: `/v3/projects/f2229dade5d84655b93f1089dc039cef` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:16.450635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2229dade5d84655b93f1089dc039cef)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:16.452600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:16.635143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2229dade5d84655b93f1089dc039cef', 'request_id': 'req-274756aa-0dec-4b71-8fb6-503cad2cf850'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:16.665603 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:16.779830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2229dade5d84655b93f1089dc039cef', 'request_id': 'req-274756aa-0dec-4b71-8fb6-503cad2cf850'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:16.786833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2229dade5d84655b93f1089dc039cef', 'request_id': 'req-274756aa-0dec-4b71-8fb6-503cad2cf850'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:16.801353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] The token cache is being invalidate because project f2229dade5d84655b93f1089dc039cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:16.801889 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:16.803355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:16.808748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1152/2299] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:16 2022] DELETE /identity/v3/projects/f2229dade5d84655b93f1089dc039cef => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:16.810189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:16.823192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79729c-1070-4b12-848a-8b169664399d service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:16.901577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:16.903056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:16.903570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:16.904075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] PATH_INFO: `/v3/users/710c72b931b8422abec9c87ea7ba6feb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:16.906436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=710c72b931b8422abec9c87ea7ba6feb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:16.910355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:17.212404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710c72b931b8422abec9c87ea7ba6feb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:17.225022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710c72b931b8422abec9c87ea7ba6feb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:17.242072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710c72b931b8422abec9c87ea7ba6feb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:17.380051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710c72b931b8422abec9c87ea7ba6feb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:17.381275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-93e67cac-e322-4c5f-b915-7a3355518f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710c72b931b8422abec9c87ea7ba6feb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:17.391726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1148/2300] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:16 2022] DELETE /identity/v3/users/710c72b931b8422abec9c87ea7ba6feb => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:17.831507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:17.953350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:17.955093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:17.973746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274756aa-0dec-4b71-8fb6-503cad2cf850 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:18.055566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:18.057510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.058078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.058588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] PATH_INFO: `/v3/projects/8688bfab37c44c2096dbc3616418bd98` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.060864 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8688bfab37c44c2096dbc3616418bd98)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:18.064809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:18.106386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:18.193432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8688bfab37c44c2096dbc3616418bd98', 'request_id': 'req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:18.293929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8688bfab37c44c2096dbc3616418bd98', 'request_id': 'req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:18.301113 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8688bfab37c44c2096dbc3616418bd98', 'request_id': 'req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:18.317341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] The token cache is being invalidate because project 8688bfab37c44c2096dbc3616418bd98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:18.317341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:18.317341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcfbf53b-6676-4da9-9303-cfaa6f6e3c8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:18.317341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1153/2301] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:17 2022] DELETE /identity/v3/projects/8688bfab37c44c2096dbc3616418bd98 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:18.321061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79358a3c-79ea-468b-b1f0-c826ba1b8cb8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.321356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79358a3c-79ea-468b-b1f0-c826ba1b8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.321732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79358a3c-79ea-468b-b1f0-c826ba1b8cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.340313 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-79358a3c-79ea-468b-b1f0-c826ba1b8cb8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:18.341832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-79358a3c-79ea-468b-b1f0-c826ba1b8cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:18.411206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1149/2302] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:18.419753 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f70433-b6e2-47ac-9575-e17805d77e01 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.420067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f70433-b6e2-47ac-9575-e17805d77e01 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.420383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f70433-b6e2-47ac-9575-e17805d77e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.439938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36f70433-b6e2-47ac-9575-e17805d77e01 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:18.441468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36f70433-b6e2-47ac-9575-e17805d77e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:18.453101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1154/2303] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:18.454263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79358a3c-79ea-468b-b1f0-c826ba1b8cb8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:18.465325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79358a3c-79ea-468b-b1f0-c826ba1b8cb8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:18.470448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:18.471127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.471416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.471695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.472442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:18.472757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:18.474413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:18.488529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1150/2304] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:53:18.497269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f70433-b6e2-47ac-9575-e17805d77e01 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:18.507120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f70433-b6e2-47ac-9575-e17805d77e01 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:18.512194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:18.512863 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.513150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.513427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.514138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:18.516838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:18.678851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1155/2305] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:18 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:18.683110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:18.694066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a425b5a-4cbe-4b38-8f7a-cc71d5c0531e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:18.751407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:18.752086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.752376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.752656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.753399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:18.755400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:18.901227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1151/2306] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:18 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:18.905356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:18.916539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55822ca-e1d5-4fee-83d4-86a0bc377a2e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:18.921572 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:18.922251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.922569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.923064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.923813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:18.925651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:18.941131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1156/2307] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:18.944771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:18.955452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383155f4-34ad-41af-9a09-306842ff868d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:18.960989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6cead-73b0-49d4-9341-2b6b12047921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:18.961676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6cead-73b0-49d4-9341-2b6b12047921 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.962084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6cead-73b0-49d4-9341-2b6b12047921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:18.962362 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6cead-73b0-49d4-9341-2b6b12047921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:18.963102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6cead-73b0-49d4-9341-2b6b12047921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:18.964721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6cead-73b0-49d4-9341-2b6b12047921 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:18.979265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1152/2308] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:18.980478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:18.991016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8efd6-9c6a-4835-af9b-982151e19653 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:18.996438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:18.997701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:18.999158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:18.999722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.000084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] PATH_INFO: `/v3/projects/c365d7f769c944b59a7bbb09d1ea998e/users/845aa6791cbc4a538b38d9e78cb58bfe/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.016818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c365d7f769c944b59a7bbb09d1ea998e, user_id=845aa6791cbc4a538b38d9e78cb58bfe, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:19.019734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:19.157579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1157/2309] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:18 2022] PUT /identity/v3/projects/c365d7f769c944b59a7bbb09d1ea998e/users/845aa6791cbc4a538b38d9e78cb58bfe/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:19.169956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a283ea-c682-4faf-a2d7-34d5161e30c1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:19.170520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a283ea-c682-4faf-a2d7-34d5161e30c1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.170520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a283ea-c682-4faf-a2d7-34d5161e30c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.200723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f8a283ea-c682-4faf-a2d7-34d5161e30c1 None None] MFA Rules not processed for user `845aa6791cbc4a538b38d9e78cb58bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:19.201979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f8a283ea-c682-4faf-a2d7-34d5161e30c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:19.274644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1153/2310] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:19.279204 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:19.347046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075f49-1eb0-4e19-a48f-84ca3cfe847c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:19.353194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:19.353874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:19.354159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.354432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.355134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:19.357188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:19.524718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1158/2311] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:19 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:19.529431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a283ea-c682-4faf-a2d7-34d5161e30c1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:19.541627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:19.542902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a283ea-c682-4faf-a2d7-34d5161e30c1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:19.605043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:19.605835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:19.606374 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.606645 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.608530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:19.609478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:19.700678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1154/2312] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:19 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:19.705142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:19.718722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f33cb-21c5-4979-af23-8fb4807b32f6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:19.725896 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:19.726881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:19.727236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.727523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.728327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:19.731251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:19.743168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1159/2313] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:19.748078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:19.761598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ccb0e-af1e-42d4-a7f5-20dcaf2dadf5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:19.768570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:19.769651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:19.770009 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.770288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] PATH_INFO: `/v3/projects/c6933a18a6b24f9a93c5247556979ce0/users/01f88234e20443cda5deee36f82b8b3a/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.786956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6933a18a6b24f9a93c5247556979ce0, user_id=01f88234e20443cda5deee36f82b8b3a, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:19.790759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:19.934993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1155/2314] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:19 2022] PUT /identity/v3/projects/c6933a18a6b24f9a93c5247556979ce0/users/01f88234e20443cda5deee36f82b8b3a/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:19.939033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:19.950177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09371299-2ac3-436e-a1ca-efa3c39da251 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:20.010722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b443aa7-312e-42d9-8168-b72190f08638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:20.011524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443aa7-312e-42d9-8168-b72190f08638 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:20.011917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443aa7-312e-42d9-8168-b72190f08638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:20.012303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443aa7-312e-42d9-8168-b72190f08638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:20.013403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b443aa7-312e-42d9-8168-b72190f08638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:20.015265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b443aa7-312e-42d9-8168-b72190f08638 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:20.026701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1160/2315] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:20.030909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:20.042326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9bac-fbd2-40ad-8aa0-2ee167390f9d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:20.042578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:20.047218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:20.052565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:20.052565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:20.052565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] PATH_INFO: `/v3/projects/c6933a18a6b24f9a93c5247556979ce0/users/01f88234e20443cda5deee36f82b8b3a/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:20.052565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6933a18a6b24f9a93c5247556979ce0, user_id=01f88234e20443cda5deee36f82b8b3a, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:20.054081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:20.065729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1156/2316] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:20 2022] PUT /identity/v3/projects/c6933a18a6b24f9a93c5247556979ce0/users/01f88234e20443cda5deee36f82b8b3a/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:20.077189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85319d55-09a5-4f84-9904-8a73a32e560d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:20.077477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85319d55-09a5-4f84-9904-8a73a32e560d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:20.077755 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85319d55-09a5-4f84-9904-8a73a32e560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:20.108124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-85319d55-09a5-4f84-9904-8a73a32e560d None None] MFA Rules not processed for user `01f88234e20443cda5deee36f82b8b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:20.109368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-85319d55-09a5-4f84-9904-8a73a32e560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:20.184647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1161/2317] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:20.197956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:20.199790 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:20.262629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220ef3f-b72d-4612-b67f-e7ba70233cbe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:20.266372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:20.267229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:20.267371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:20.267568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:20.277237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:20.278666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:20.288971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1157/2318] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:20.408310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:21.954988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85319d55-09a5-4f84-9904-8a73a32e560d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:21.959756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85319d55-09a5-4f84-9904-8a73a32e560d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:21.966911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f88234e20443cda5deee36f82b8b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6933a18a6b24f9a93c5247556979ce0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:21.967819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:21.968315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:21.968602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:21.969613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:21.970001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:21.971893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:21.991185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1162/2319] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:53:21.995857 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:22.006925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b33cf-0718-49f9-833e-2dc62b695af1 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:22.013197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f88234e20443cda5deee36f82b8b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6933a18a6b24f9a93c5247556979ce0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:22.013888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:22.014191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:22.014721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:22.015774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:22.017845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:22.157177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1158/2320] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:21 2022] POST /identity/v3/projects => generated 354 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:22.158344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:22.169563 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7cb05-3258-4c05-9d30-7a236902f773 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:22.233297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:22.240992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f88234e20443cda5deee36f82b8b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6933a18a6b24f9a93c5247556979ce0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:22.242139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:22.242429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:22.242709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:22.244086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:22.244406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:22.246135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:22.261549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1163/2321] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:53:22.303748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:22.315650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01358d35-cdbd-4d55-aa23-d1cc83596de0 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:22.321519 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f88234e20443cda5deee36f82b8b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6933a18a6b24f9a93c5247556979ce0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:22.322445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:22.322733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:22.323014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] PATH_INFO: `/v3/projects/51d6f2b77a79448ba0da1cab3af9c194` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:22.324247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorizing `identity:delete_project(project_id=51d6f2b77a79448ba0da1cab3af9c194)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:22.326268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:22.514666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d6f2b77a79448ba0da1cab3af9c194', 'request_id': 'req-95379b05-75ad-42e3-afcd-ca860dd7ec30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:22.522595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d6f2b77a79448ba0da1cab3af9c194', 'request_id': 'req-95379b05-75ad-42e3-afcd-ca860dd7ec30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:22.608459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d6f2b77a79448ba0da1cab3af9c194', 'request_id': 'req-95379b05-75ad-42e3-afcd-ca860dd7ec30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:22.620805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] The token cache is being invalidate because project 51d6f2b77a79448ba0da1cab3af9c194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:22.621121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:22.621779 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95379b05-75ad-42e3-afcd-ca860dd7ec30 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:22.623355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1159/2322] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:22 2022] DELETE /identity/v3/projects/51d6f2b77a79448ba0da1cab3af9c194 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:23.539777 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:23.541635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:23.610010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a51906-383b-40ff-89e8-fe0e3801b9e3 tempest-VolumeQuotaClassesTest-1794913934 tempest-VolumeQuotaClassesTest-1794913934-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:23.673558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:23.674545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:23.674825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:23.675098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] PATH_INFO: `/v3/users/845aa6791cbc4a538b38d9e78cb58bfe` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:23.676300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=845aa6791cbc4a538b38d9e78cb58bfe)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:23.678571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.005333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '845aa6791cbc4a538b38d9e78cb58bfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.018432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '845aa6791cbc4a538b38d9e78cb58bfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.173580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '845aa6791cbc4a538b38d9e78cb58bfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.174193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '845aa6791cbc4a538b38d9e78cb58bfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.174842 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '845aa6791cbc4a538b38d9e78cb58bfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.188682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1164/2323] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:23 2022] DELETE /identity/v3/users/845aa6791cbc4a538b38d9e78cb58bfe => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:24.193817 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.194383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.194477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.212701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:24.213978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:24.223088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:24.276569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1160/2324] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:24.295756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.297232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:24.319851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:24.359369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375e5d-b2e9-4a53-8a31-e9dce43d8ff3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.363850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.364439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.364730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.365041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.375729 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.377242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.388249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1165/2325] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:24.987341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.990497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:25.020081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf38897-021c-41c7-9ade-2547f43b76ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:25.026710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:25.027621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:25.027906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:25.028208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] PATH_INFO: `/v3/projects/c365d7f769c944b59a7bbb09d1ea998e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:25.029438 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c365d7f769c944b59a7bbb09d1ea998e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:25.031379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:25.175379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c365d7f769c944b59a7bbb09d1ea998e', 'request_id': 'req-f78288d1-9dec-4c79-8425-0f5c923214f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.182205 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c365d7f769c944b59a7bbb09d1ea998e', 'request_id': 'req-f78288d1-9dec-4c79-8425-0f5c923214f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.324589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c365d7f769c944b59a7bbb09d1ea998e', 'request_id': 'req-f78288d1-9dec-4c79-8425-0f5c923214f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.336745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] The token cache is being invalidate because project c365d7f769c944b59a7bbb09d1ea998e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:25.337064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.337733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.339384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1161/2326] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:24 2022] DELETE /identity/v3/projects/c365d7f769c944b59a7bbb09d1ea998e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:25.343710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:25.353904 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138db086-0f3a-4b37-8f6e-10e89b260bd5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:25.417911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:25.418946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:25.419297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:25.419578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] PATH_INFO: `/v3/users/01f88234e20443cda5deee36f82b8b3a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:25.420825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f88234e20443cda5deee36f82b8b3a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:25.422760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:25.692234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f88234e20443cda5deee36f82b8b3a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.705756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f88234e20443cda5deee36f82b8b3a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.820719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f88234e20443cda5deee36f82b8b3a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.821475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f88234e20443cda5deee36f82b8b3a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.822205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aaa87f69-7a02-4d47-865d-2eb9a6399649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f88234e20443cda5deee36f82b8b3a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.835696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1166/2327] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:25 2022] DELETE /identity/v3/users/01f88234e20443cda5deee36f82b8b3a => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:26.357403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:26.358977 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:26.374041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78288d1-9dec-4c79-8425-0f5c923214f2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:26.439968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:26.440876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:26.441162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:26.441439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] PATH_INFO: `/v3/projects/c6933a18a6b24f9a93c5247556979ce0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:26.448995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6933a18a6b24f9a93c5247556979ce0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:26.448995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:26.587136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:26.594765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6933a18a6b24f9a93c5247556979ce0', 'request_id': 'req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.602437 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6933a18a6b24f9a93c5247556979ce0', 'request_id': 'req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.717968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6933a18a6b24f9a93c5247556979ce0', 'request_id': 'req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.733397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] The token cache is being invalidate because project c6933a18a6b24f9a93c5247556979ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:26.733701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.734547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62374f51-b89a-4413-8c97-0ab7bbf3dbc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.736309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1162/2328] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:26 2022] DELETE /identity/v3/projects/c6933a18a6b24f9a93c5247556979ce0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:26.742972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75d6d-ac49-444d-8138-880d374ab964 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:26.743543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75d6d-ac49-444d-8138-880d374ab964 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:26.743759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75d6d-ac49-444d-8138-880d374ab964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:26.766841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-46f75d6d-ac49-444d-8138-880d374ab964 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:26.767662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-46f75d6d-ac49-444d-8138-880d374ab964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:26.831184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:26.878869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1167/2329] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:26.888307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d98df-a1e0-44a0-b93f-d42daabc4ec7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:26.888838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d98df-a1e0-44a0-b93f-d42daabc4ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:26.889757 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d98df-a1e0-44a0-b93f-d42daabc4ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:26.909149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1a9d98df-a1e0-44a0-b93f-d42daabc4ec7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:26.910418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1a9d98df-a1e0-44a0-b93f-d42daabc4ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:26.920214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1163/2330] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:26.924606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75d6d-ac49-444d-8138-880d374ab964 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:26.935972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75d6d-ac49-444d-8138-880d374ab964 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:26.940713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:26.941436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:26.941962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:26.942161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:26.942903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:26.943221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:26.945625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:26.958917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1168/2331] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:53:26.962992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d98df-a1e0-44a0-b93f-d42daabc4ec7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:26.972693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d98df-a1e0-44a0-b93f-d42daabc4ec7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:26.978009 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:26.978669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:26.978957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:26.979237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:26.979979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:26.981958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:27.135899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1164/2332] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:26 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:27.136101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:27.145761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2ab28-50c1-49c0-9fec-051734894e51 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:27.216636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:27.217354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:27.222062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:27.222062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:27.222062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:27.222062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:27.436401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1169/2333] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:27 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:27.436629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:27.436629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3a25-8e8b-481a-9191-05fa9beb2dc0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:27.436629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:27.436629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:27.437372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:27.437372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:27.437372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:27.437372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:27.439557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1165/2334] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:27.444949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:27.455906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2308eb-37b7-4630-b6f4-5be434934815 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:27.462435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad3276-206a-4b48-99b7-0b0431a2ef12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:27.463155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad3276-206a-4b48-99b7-0b0431a2ef12 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:27.463482 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad3276-206a-4b48-99b7-0b0431a2ef12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:27.463859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad3276-206a-4b48-99b7-0b0431a2ef12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:27.464727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad3276-206a-4b48-99b7-0b0431a2ef12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:27.467132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad3276-206a-4b48-99b7-0b0431a2ef12 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:27.479704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1170/2335] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:27.502087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:27.522253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb1f44-8ce1-4af5-9be5-262b5b80428f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:27.527889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:27.528845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:27.529138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:27.529416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] PATH_INFO: `/v3/projects/560a1ab919534cb7904bb253527f21bd/users/0be9cc2a487f47edb2c4a252eab839b8/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:27.546220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=560a1ab919534cb7904bb253527f21bd, user_id=0be9cc2a487f47edb2c4a252eab839b8, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:27.548918 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:27.813337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1166/2336] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:27 2022] PUT /identity/v3/projects/560a1ab919534cb7904bb253527f21bd/users/0be9cc2a487f47edb2c4a252eab839b8/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:27.829092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a258c0-bfaf-490b-bb19-3afa30da391e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:27.835491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a258c0-bfaf-490b-bb19-3afa30da391e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:27.835491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a258c0-bfaf-490b-bb19-3afa30da391e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:27.861273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-95a258c0-bfaf-490b-bb19-3afa30da391e None None] MFA Rules not processed for user `0be9cc2a487f47edb2c4a252eab839b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:27.863087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95a258c0-bfaf-490b-bb19-3afa30da391e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:27.936536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1171/2337] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:27.942569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:28.013358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e6ca-4359-4a8b-9c39-68aa2e5e48fc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:28.018830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:28.019543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:28.019834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:28.020122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:28.020868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:28.023474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:28.175323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1167/2338] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:27 2022] POST /identity/v3/projects => generated 379 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:28.180039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a258c0-bfaf-490b-bb19-3afa30da391e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:28.191382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a258c0-bfaf-490b-bb19-3afa30da391e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:28.227597 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:28.251894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:28.252680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:28.252999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:28.253285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:28.254021 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:28.255970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:28.407249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1172/2339] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:28 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:28.408609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:28.419856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8506e-b54a-46d0-b405-dbef4c0e0bf2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:28.425034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:28.425718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:28.426069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:28.426352 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:28.427157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:28.429185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:28.445241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1168/2340] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:28.449125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:28.461692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d97191-5983-4581-88a1-5ec1eb3cfeb4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:28.468571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd708a8-66e4-40a2-9dc8-f12e573ed2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:28.470071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd708a8-66e4-40a2-9dc8-f12e573ed2de admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:28.470071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd708a8-66e4-40a2-9dc8-f12e573ed2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:28.470301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd708a8-66e4-40a2-9dc8-f12e573ed2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:28.471236 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd708a8-66e4-40a2-9dc8-f12e573ed2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:28.473094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd708a8-66e4-40a2-9dc8-f12e573ed2de admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:28.484581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1173/2341] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:28.488834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:28.500347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9220b93-9f1e-4589-845f-36a623f17d36 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:28.505383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:28.507120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:28.507495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:28.507789 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] PATH_INFO: `/v3/projects/302d3c9ee85947008e8e49c43f962be2/users/b24238e6e0c04d14babe9087d82b4ae6/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:28.527495 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=302d3c9ee85947008e8e49c43f962be2, user_id=b24238e6e0c04d14babe9087d82b4ae6, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:28.530257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:28.666146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1169/2342] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:28 2022] PUT /identity/v3/projects/302d3c9ee85947008e8e49c43f962be2/users/b24238e6e0c04d14babe9087d82b4ae6/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:28.676800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c397b14-d6a2-475e-b139-ac3b3b710da1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:28.677093 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c397b14-d6a2-475e-b139-ac3b3b710da1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:28.677390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c397b14-d6a2-475e-b139-ac3b3b710da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:28.714370 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c397b14-d6a2-475e-b139-ac3b3b710da1 None None] MFA Rules not processed for user `b24238e6e0c04d14babe9087d82b4ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:28.715832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c397b14-d6a2-475e-b139-ac3b3b710da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:28.788631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1174/2343] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:28.793202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:28.811685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:28.864298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a420f5a-6da2-41de-a62d-3cc5ecda2314 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:28.870177 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:28.870843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:28.871283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:28.871573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:28.872292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:28.879103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:28.977288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1170/2344] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:28 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:28.981838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c397b14-d6a2-475e-b139-ac3b3b710da1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:28.993530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c397b14-d6a2-475e-b139-ac3b3b710da1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:29.053769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:29.054372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.054660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.054981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.055679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:29.057822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:29.185740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1175/2345] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:28 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:29.189972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:29.201960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd46974-45c8-474e-81ce-7b2822e44295 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:29.212425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:29.213155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.213444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.213723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.214470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:29.216359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:29.227894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1171/2346] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:29.232109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:29.243903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612eadab-9a2f-433c-b6ce-c169d17763fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:29.248651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:29.249583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.249861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.250135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] PATH_INFO: `/v3/projects/c055de94913a4ba48de2d2fa51e5fb58/users/d608064329b942d5bab0be9e70fe022f/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.267710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c055de94913a4ba48de2d2fa51e5fb58, user_id=d608064329b942d5bab0be9e70fe022f, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:29.270631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:29.418895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1176/2347] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:29 2022] PUT /identity/v3/projects/c055de94913a4ba48de2d2fa51e5fb58/users/d608064329b942d5bab0be9e70fe022f/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:29.419010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:29.425222 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73469205-89d9-4f9c-91e4-1274d9735209 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:29.485660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf79ce3-cae1-4d69-a7c6-430b9fa41aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:29.486474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79ce3-cae1-4d69-a7c6-430b9fa41aff admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.486765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79ce3-cae1-4d69-a7c6-430b9fa41aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.487045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79ce3-cae1-4d69-a7c6-430b9fa41aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.487774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf79ce3-cae1-4d69-a7c6-430b9fa41aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:29.489408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf79ce3-cae1-4d69-a7c6-430b9fa41aff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:29.499819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1172/2348] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:29.504125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:29.515005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52debb74-196f-4a16-a2e3-540ef428f570 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:29.544866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:29.544866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.544866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.544866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] PATH_INFO: `/v3/projects/c055de94913a4ba48de2d2fa51e5fb58/users/d608064329b942d5bab0be9e70fe022f/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.545521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c055de94913a4ba48de2d2fa51e5fb58, user_id=d608064329b942d5bab0be9e70fe022f, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:29.545521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:29.545521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:29.545521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1177/2349] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:29 2022] PUT /identity/v3/projects/c055de94913a4ba48de2d2fa51e5fb58/users/d608064329b942d5bab0be9e70fe022f/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:29.555750 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.556051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.556355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.585253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] MFA Rules not processed for user `d608064329b942d5bab0be9e70fe022f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:29.596234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:29.673017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1173/2350] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:29.685385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:29.687455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:29.749310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2826b-0898-4d22-a790-1ea903e2b050 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:29.753237 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6be4eb-6af8-48dd-9395-936fa1923380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:29.753892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be4eb-6af8-48dd-9395-936fa1923380 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.754178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be4eb-6af8-48dd-9395-936fa1923380 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.754518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be4eb-6af8-48dd-9395-936fa1923380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.765872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be4eb-6af8-48dd-9395-936fa1923380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:29.768599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be4eb-6af8-48dd-9395-936fa1923380 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:29.778128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1178/2351] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:29.954371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:29.955947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:29.969116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200d8a5-b8ac-422d-8a62-165538ad977d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:29.972975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:29.973651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:29.973968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:29.974247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:29.981250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:30.040998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:30.042563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:30.051934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1174/2352] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:31.147668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f2a99-5f5e-427c-bfaa-d778d775a587 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:31.148238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f2a99-5f5e-427c-bfaa-d778d775a587 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:31.148461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f2a99-5f5e-427c-bfaa-d778d775a587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:31.182826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-027f2a99-5f5e-427c-bfaa-d778d775a587 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:31.184249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-027f2a99-5f5e-427c-bfaa-d778d775a587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:31.262739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1179/2353] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:53:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:31.641931 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:31.644012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:31.723694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4badea-0e97-44b3-8021-16a6526b1f12 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:31.729220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e700c-6005-41df-a5dd-ec54b6c5fe49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:31.730004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e700c-6005-41df-a5dd-ec54b6c5fe49 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:31.730735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e700c-6005-41df-a5dd-ec54b6c5fe49 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:31.730735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e700c-6005-41df-a5dd-ec54b6c5fe49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:31.739652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e700c-6005-41df-a5dd-ec54b6c5fe49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:31.742164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e700c-6005-41df-a5dd-ec54b6c5fe49 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:31.751972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1175/2354] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:53:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:53:35.056456 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f2a99-5f5e-427c-bfaa-d778d775a587 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:35.164437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f2a99-5f5e-427c-bfaa-d778d775a587 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:35.174307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:35.175355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:35.175695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:35.176048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] PATH_INFO: `/v3/users/0be9cc2a487f47edb2c4a252eab839b8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:35.178637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be9cc2a487f47edb2c4a252eab839b8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:35.180836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:35.682530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:35.692797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be9cc2a487f47edb2c4a252eab839b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:35.708359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be9cc2a487f47edb2c4a252eab839b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:35.953147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be9cc2a487f47edb2c4a252eab839b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:35.954102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be9cc2a487f47edb2c4a252eab839b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:35.955765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be9cc2a487f47edb2c4a252eab839b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:35.997369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1180/2355] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:35 2022] DELETE /identity/v3/users/0be9cc2a487f47edb2c4a252eab839b8 => generated 0 bytes in 936 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:35.997606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:35.997606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:35.997606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:36.038261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:36.039348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:36.156677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1176/2356] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:36.169431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:36.171074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:36.244623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e10d8-2278-4b17-9480-e21eb6b12de0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:36.249373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:36.250124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:36.250483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:36.250848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:36.265766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:36.267658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:36.284498 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1181/2357] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:37.107608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:37.112416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:37.151994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76171a-af0c-4d15-8b87-3a45108e2373 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:37.158686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:37.159742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:37.159891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:37.160228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] PATH_INFO: `/v3/projects/560a1ab919534cb7904bb253527f21bd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:37.162989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=560a1ab919534cb7904bb253527f21bd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:37.166114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:37.335835 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:37.353904 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560a1ab919534cb7904bb253527f21bd', 'request_id': 'req-baf60319-92aa-4bf4-b645-9a896c895caf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.362285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560a1ab919534cb7904bb253527f21bd', 'request_id': 'req-baf60319-92aa-4bf4-b645-9a896c895caf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.501357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560a1ab919534cb7904bb253527f21bd', 'request_id': 'req-baf60319-92aa-4bf4-b645-9a896c895caf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.531733 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] The token cache is being invalidate because project 560a1ab919534cb7904bb253527f21bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:37.532450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.533257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.535145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1177/2358] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:37 2022] DELETE /identity/v3/projects/560a1ab919534cb7904bb253527f21bd => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:37.542470 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:37.558584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052fa4e-f78a-4a07-a786-d1f42dc3139e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:37.642269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:37.643699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:37.644315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:37.644843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] PATH_INFO: `/v3/users/b24238e6e0c04d14babe9087d82b4ae6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:37.646706 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24238e6e0c04d14babe9087d82b4ae6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:37.649119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:37.988903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24238e6e0c04d14babe9087d82b4ae6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.000803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:38.023319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24238e6e0c04d14babe9087d82b4ae6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.094159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24238e6e0c04d14babe9087d82b4ae6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.094912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24238e6e0c04d14babe9087d82b4ae6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.095616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24238e6e0c04d14babe9087d82b4ae6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.114057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1182/2359] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:37 2022] DELETE /identity/v3/users/b24238e6e0c04d14babe9087d82b4ae6 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:38.845294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:38.847807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:38.879209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60319-92aa-4bf4-b645-9a896c895caf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:38.950269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:38.951165 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:38.951451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:38.951727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] PATH_INFO: `/v3/projects/302d3c9ee85947008e8e49c43f962be2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:38.953215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] RBAC: Authorizing `identity:delete_project(project_id=302d3c9ee85947008e8e49c43f962be2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:38.955185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:39.353521 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302d3c9ee85947008e8e49c43f962be2', 'request_id': 'req-dd711be5-1bf6-4478-90bd-0b109b7dedca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:39.360644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302d3c9ee85947008e8e49c43f962be2', 'request_id': 'req-dd711be5-1bf6-4478-90bd-0b109b7dedca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:39.499159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302d3c9ee85947008e8e49c43f962be2', 'request_id': 'req-dd711be5-1bf6-4478-90bd-0b109b7dedca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:39.516417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] The token cache is being invalidate because project 302d3c9ee85947008e8e49c43f962be2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:39.516894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:39.517712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:39.519865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1178/2360] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:38 2022] DELETE /identity/v3/projects/302d3c9ee85947008e8e49c43f962be2 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:39.525775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:39.527883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:39.557521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13be6ec-0704-4c95-88fd-eb4ec9848edd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:39.641659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:39.642619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:39.643001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:39.643363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] PATH_INFO: `/v3/users/d608064329b942d5bab0be9e70fe022f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:39.644655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d608064329b942d5bab0be9e70fe022f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:39.648307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:39.964200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd608064329b942d5bab0be9e70fe022f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:39.993474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd608064329b942d5bab0be9e70fe022f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:40.076599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd608064329b942d5bab0be9e70fe022f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:40.077770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd608064329b942d5bab0be9e70fe022f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:40.080074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd608064329b942d5bab0be9e70fe022f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:40.128737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1183/2361] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:39 2022] DELETE /identity/v3/users/d608064329b942d5bab0be9e70fe022f => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:40.856895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:40.858620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:40.973332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd711be5-1bf6-4478-90bd-0b109b7dedca admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:40.984518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:41.075455 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eb309-92dd-4b8b-be3b-40e3476ddaa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:41.082344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb309-92dd-4b8b-be3b-40e3476ddaa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:41.082344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb309-92dd-4b8b-be3b-40e3476ddaa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:41.082344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb309-92dd-4b8b-be3b-40e3476ddaa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:41.084169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:41.086173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:41.086446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f74eb309-92dd-4b8b-be3b-40e3476ddaa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:41.221536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:41.229275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa50f4-80a9-4263-bdca-0e9169e527ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:41.233998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eb309-92dd-4b8b-be3b-40e3476ddaa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:41.234536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:41.235435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:41.235716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:41.236091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] PATH_INFO: `/v3/projects/c055de94913a4ba48de2d2fa51e5fb58` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:41.238515 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eb309-92dd-4b8b-be3b-40e3476ddaa4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:41.240591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c055de94913a4ba48de2d2fa51e5fb58)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:41.242690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:41.260568 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1179/2362] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:41.466027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c055de94913a4ba48de2d2fa51e5fb58', 'request_id': 'req-f7227a7a-86f3-479e-8162-63cae099ed8d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:41.636980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c055de94913a4ba48de2d2fa51e5fb58', 'request_id': 'req-f7227a7a-86f3-479e-8162-63cae099ed8d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:41.647104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c055de94913a4ba48de2d2fa51e5fb58', 'request_id': 'req-f7227a7a-86f3-479e-8162-63cae099ed8d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:41.669497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] The token cache is being invalidate because project c055de94913a4ba48de2d2fa51e5fb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:41.670128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:41.671047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f7227a7a-86f3-479e-8162-63cae099ed8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:41.673776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1184/2363] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:41 2022] DELETE /identity/v3/projects/c055de94913a4ba48de2d2fa51e5fb58 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:41.679771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2799f-6894-401a-a149-9f047968e136 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:41.680071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2799f-6894-401a-a149-9f047968e136 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:41.680458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2799f-6894-401a-a149-9f047968e136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:41.719989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-90d2799f-6894-401a-a149-9f047968e136 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:41.722988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-90d2799f-6894-401a-a149-9f047968e136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:41.876292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1180/2364] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:41.885680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb297c-f096-4d73-bd9f-2fc88015091f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:41.886769 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb297c-f096-4d73-bd9f-2fc88015091f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:41.887414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb297c-f096-4d73-bd9f-2fc88015091f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:41.952386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-98fb297c-f096-4d73-bd9f-2fc88015091f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:41.954797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-98fb297c-f096-4d73-bd9f-2fc88015091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:42.013407 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1185/2365] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:42.017992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2799f-6894-401a-a149-9f047968e136 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:42.035687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2799f-6894-401a-a149-9f047968e136 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:42.052130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:42.053302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:42.053634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:42.054019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:42.055851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:42.056295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:42.063394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:42.100429 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1181/2366] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:53:42.104225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb297c-f096-4d73-bd9f-2fc88015091f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:42.123887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb297c-f096-4d73-bd9f-2fc88015091f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:42.135588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:42.136979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:42.143950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:42.143950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:42.143950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:42.143950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:42.295444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1186/2367] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:42 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:42.299410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:42.320899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c16f1-52f7-4701-9ed6-9a4f674253e7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:42.381660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:42.406750 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:42.407518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:42.407882 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:42.408816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:42.410657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:42.412855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:42.579197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1182/2368] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:42 2022] POST /identity/v3/users => generated 291 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:42.586125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:42.606313 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243fd69-4d78-478a-9709-72107b44db74 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:42.612673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:42.613531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:42.613900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:42.614302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:42.615882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:42.618047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:42.632824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1187/2369] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:42.636870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:42.652239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f384-71bb-4e2c-a0ef-f0ce6b54e5a8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:42.657530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3f387-d299-44fc-bd7b-c4dc18d78293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:42.658308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3f387-d299-44fc-bd7b-c4dc18d78293 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:42.658672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3f387-d299-44fc-bd7b-c4dc18d78293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:42.659017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3f387-d299-44fc-bd7b-c4dc18d78293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:42.659834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3f387-d299-44fc-bd7b-c4dc18d78293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:42.661797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3f387-d299-44fc-bd7b-c4dc18d78293 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:42.676164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1183/2370] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:42.680373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:42.690941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88b678-072c-4fa3-a145-f32c226e2f40 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:42.701088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:42.704164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:42.704164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:42.704164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] PATH_INFO: `/v3/projects/ae6973b53f194d3a9fa293bc95276a71/users/5611409ef28744fca2f395bba76908c6/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:42.722682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6973b53f194d3a9fa293bc95276a71, user_id=5611409ef28744fca2f395bba76908c6, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:42.726981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:42.824400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1188/2371] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:42 2022] PUT /identity/v3/projects/ae6973b53f194d3a9fa293bc95276a71/users/5611409ef28744fca2f395bba76908c6/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:42.835871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135f1d4-aa6c-4254-b263-dbd10fdf8147 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:42.846763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135f1d4-aa6c-4254-b263-dbd10fdf8147 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:42.846763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135f1d4-aa6c-4254-b263-dbd10fdf8147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:42.867845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2135f1d4-aa6c-4254-b263-dbd10fdf8147 None None] MFA Rules not processed for user `5611409ef28744fca2f395bba76908c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:42.869303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2135f1d4-aa6c-4254-b263-dbd10fdf8147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:43.001646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1184/2372] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:43.005816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:43.169691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e25f25-8883-439e-a9b3-bb28855cf6c6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:43.177476 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:43.178174 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:43.178461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:43.178744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:43.179528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:43.181578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:43.353252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1189/2373] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:43 2022] POST /identity/v3/projects => generated 379 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:43.353702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135f1d4-aa6c-4254-b263-dbd10fdf8147 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:43.367651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135f1d4-aa6c-4254-b263-dbd10fdf8147 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:43.439777 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:43.440528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:43.440865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:43.441148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:43.441898 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:43.445173 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:43.689716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1185/2374] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:43 2022] POST /identity/v3/users => generated 296 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:43.696474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:43.716922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989085a3-9b4d-481e-b0eb-9a94574b0351 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:43.723850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:43.724934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:43.725499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:43.726065 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:43.727222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:43.729968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:43.732131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:43.748993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1190/2375] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:43.753414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:43.766266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059587ba-fa7c-480e-bec3-ee4b502c4e0d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:43.772287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:43.779067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:43.779067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:43.779067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] PATH_INFO: `/v3/projects/5bd5e8591859498daad7b79584dfbc0a/users/e3e51d87532447baa844a6ab97b2a3ba/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:43.793477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd5e8591859498daad7b79584dfbc0a, user_id=e3e51d87532447baa844a6ab97b2a3ba, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:43.795051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:43.924168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1186/2376] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:43 2022] PUT /identity/v3/projects/5bd5e8591859498daad7b79584dfbc0a/users/e3e51d87532447baa844a6ab97b2a3ba/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:43.931047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:43.955506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055dff6-c33f-473a-81c0-50411da20c56 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:44.061107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb98e6-08c3-4582-9458-bec9b092316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:44.061850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb98e6-08c3-4582-9458-bec9b092316d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:44.062134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb98e6-08c3-4582-9458-bec9b092316d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:44.062606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb98e6-08c3-4582-9458-bec9b092316d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:44.063721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb98e6-08c3-4582-9458-bec9b092316d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:44.067291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb98e6-08c3-4582-9458-bec9b092316d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:44.080964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1191/2377] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:44.085310 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:44.098700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92022-2ccf-46d7-8f26-9daae5ffd8ff admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:44.104693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:44.105709 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:44.108208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:44.108208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] PATH_INFO: `/v3/projects/5bd5e8591859498daad7b79584dfbc0a/users/e3e51d87532447baa844a6ab97b2a3ba/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:44.109756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd5e8591859498daad7b79584dfbc0a, user_id=e3e51d87532447baa844a6ab97b2a3ba, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:44.113149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:44.126671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1187/2378] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:44 2022] PUT /identity/v3/projects/5bd5e8591859498daad7b79584dfbc0a/users/e3e51d87532447baa844a6ab97b2a3ba/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:44.201346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5a52-1187-47b3-b422-0e0e610fa3f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:44.207081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5a52-1187-47b3-b422-0e0e610fa3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:44.207081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5a52-1187-47b3-b422-0e0e610fa3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:44.240538 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a32b5a52-1187-47b3-b422-0e0e610fa3f7 None None] MFA Rules not processed for user `e3e51d87532447baa844a6ab97b2a3ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:44.242790 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a32b5a52-1187-47b3-b422-0e0e610fa3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:44.378079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1192/2379] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:44.391181 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:44.394888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:44.406497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:44.523331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb82c4-73dd-486d-9daa-d94e316b5dc1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:44.538088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb12e-2384-4703-8984-46fd518c48ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:44.538088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb12e-2384-4703-8984-46fd518c48ac service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:44.538088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb12e-2384-4703-8984-46fd518c48ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:44.538088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb12e-2384-4703-8984-46fd518c48ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:44.565562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfb12e-2384-4703-8984-46fd518c48ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:44.569436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfb12e-2384-4703-8984-46fd518c48ac service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:44.584619 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1188/2380] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:45.383720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b5a52-1187-47b3-b422-0e0e610fa3f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:45.468171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b5a52-1187-47b3-b422-0e0e610fa3f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:45.473301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:45.475680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:45.475968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:45.476246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] PATH_INFO: `/v3/users/5611409ef28744fca2f395bba76908c6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:45.477552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5611409ef28744fca2f395bba76908c6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:45.479574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:45.503151 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:45.732060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5611409ef28744fca2f395bba76908c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:45.762829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5611409ef28744fca2f395bba76908c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:45.883756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5611409ef28744fca2f395bba76908c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:45.884678 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5611409ef28744fca2f395bba76908c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:45.885507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5611409ef28744fca2f395bba76908c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:45.903460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1193/2381] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:45 2022] DELETE /identity/v3/users/5611409ef28744fca2f395bba76908c6 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:45.908155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eb173-3c54-4199-bd6a-0104eca624c6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:45.908582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eb173-3c54-4199-bd6a-0104eca624c6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:45.908582 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eb173-3c54-4199-bd6a-0104eca624c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:45.932236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c28eb173-3c54-4199-bd6a-0104eca624c6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:45.933539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c28eb173-3c54-4199-bd6a-0104eca624c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:46.057167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1189/2382] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:46.072174 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:46.078653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:46.173583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961a87d-372c-43be-b8d7-34af4e1b997a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:46.179748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac301-285f-4901-ae50-5164357ddf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:46.180907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac301-285f-4901-ae50-5164357ddf4b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:46.181194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac301-285f-4901-ae50-5164357ddf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:46.181477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac301-285f-4901-ae50-5164357ddf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:46.193513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac301-285f-4901-ae50-5164357ddf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:46.195018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac301-285f-4901-ae50-5164357ddf4b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:46.221727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1194/2383] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:46.594509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb59d7-a124-4e0f-834a-22dcce63284f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:46.595113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb59d7-a124-4e0f-834a-22dcce63284f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:46.595599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb59d7-a124-4e0f-834a-22dcce63284f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:46.597311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1190/2384] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:53:46.608552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:46.611068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:46.611524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:46.631592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:46.633228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:46.772895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1195/2385] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:53:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:46.808849 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb59d7-a124-4e0f-834a-22dcce63284f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:46.815208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb59d7-a124-4e0f-834a-22dcce63284f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:47.057837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:47.057837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:47.057837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:47.057837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:47.062149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:47.062149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:47.062149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1191/2386] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:47.138901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:47.138901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:47.138901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:47.138901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7b4f0-8ffa-4bc3-8138-52de68178842 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:47.140596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:47.140596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:47.140596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:47.140596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:47.142031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:47.142031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:47.142031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:47.142031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1196/2387] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:47.262070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:47.263597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:47.284439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3992fca-f79a-4317-bf45-432eca88fc6f service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:47.295119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:47.299314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:47.299314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:47.299314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] PATH_INFO: `/v3/users/3dbb489073d94425ad1a1b350768c428` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:47.300623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:47.301062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dbb489073d94425ad1a1b350768c428)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:47.302841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:47.309459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:47.317583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5ef7-1904-4912-a660-9a844c192750 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:47.324465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:47.327735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:47.328250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:47.328716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] PATH_INFO: `/v3/projects/ae6973b53f194d3a9fa293bc95276a71` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:47.330391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6973b53f194d3a9fa293bc95276a71)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:47.334102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:47.616056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dbb489073d94425ad1a1b350768c428'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.624312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dbb489073d94425ad1a1b350768c428'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.646123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6973b53f194d3a9fa293bc95276a71', 'request_id': 'req-d223b391-3f60-4bed-a214-95ad622592d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.657535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dbb489073d94425ad1a1b350768c428'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.784899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6973b53f194d3a9fa293bc95276a71', 'request_id': 'req-d223b391-3f60-4bed-a214-95ad622592d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.793100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6973b53f194d3a9fa293bc95276a71', 'request_id': 'req-d223b391-3f60-4bed-a214-95ad622592d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.808619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] The token cache is being invalidate because project ae6973b53f194d3a9fa293bc95276a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:47.809043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.810212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.811813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1197/2388] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:47 2022] DELETE /identity/v3/projects/ae6973b53f194d3a9fa293bc95276a71 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:47.816366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:47.819298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:47.859011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dbb489073d94425ad1a1b350768c428'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.860046 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dbb489073d94425ad1a1b350768c428'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:47.867688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1192/2389] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:47 2022] DELETE /identity/v3/users/3dbb489073d94425ad1a1b350768c428 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:47.911044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223b391-3f60-4bed-a214-95ad622592d6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:47.987799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:47.988767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:47.989140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:47.989951 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] PATH_INFO: `/v3/users/e3e51d87532447baa844a6ab97b2a3ba` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:47.990837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e51d87532447baa844a6ab97b2a3ba)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:47.992911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:48.232479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:48.234180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:48.251578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd96a52-a34e-430b-af10-8ea6a828b232 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:48.257745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:48.258622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:48.258909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:48.259269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] PATH_INFO: `/v3/projects/f32dc706424247d0ac1dc7a6aa633107` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:48.260454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32dc706424247d0ac1dc7a6aa633107)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:48.262561 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:48.264909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:48.296596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e51d87532447baa844a6ab97b2a3ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.310067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e51d87532447baa844a6ab97b2a3ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.413776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32dc706424247d0ac1dc7a6aa633107', 'request_id': 'req-44b533f7-95f0-47da-8f8b-f20f9ae27c85'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.419895 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32dc706424247d0ac1dc7a6aa633107', 'request_id': 'req-44b533f7-95f0-47da-8f8b-f20f9ae27c85'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.448664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e51d87532447baa844a6ab97b2a3ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.449308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e51d87532447baa844a6ab97b2a3ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.449909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48c0eb64-bead-43b0-9c22-98e391b7f9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e51d87532447baa844a6ab97b2a3ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.463624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1198/2390] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:47 2022] DELETE /identity/v3/users/e3e51d87532447baa844a6ab97b2a3ba => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:48.590590 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32dc706424247d0ac1dc7a6aa633107', 'request_id': 'req-44b533f7-95f0-47da-8f8b-f20f9ae27c85'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.602553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] The token cache is being invalidate because project f32dc706424247d0ac1dc7a6aa633107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:48.602843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.603505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44b533f7-95f0-47da-8f8b-f20f9ae27c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:48.605131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1193/2391] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:48 2022] DELETE /identity/v3/projects/f32dc706424247d0ac1dc7a6aa633107 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:48.610334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bcf34-d860-4c91-a8b7-9708d25a5515 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:48.610655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bcf34-d860-4c91-a8b7-9708d25a5515 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:48.610911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bcf34-d860-4c91-a8b7-9708d25a5515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:48.630330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-563bcf34-d860-4c91-a8b7-9708d25a5515 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:48.631734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-563bcf34-d860-4c91-a8b7-9708d25a5515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:48.690725 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1199/2392] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:48.698934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8bb66-4a5d-4fe2-b56e-14c22ebf372e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:48.699225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8bb66-4a5d-4fe2-b56e-14c22ebf372e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:48.699509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8bb66-4a5d-4fe2-b56e-14c22ebf372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:48.718534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b9b8bb66-4a5d-4fe2-b56e-14c22ebf372e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:48.719807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b9b8bb66-4a5d-4fe2-b56e-14c22ebf372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:48.729426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1194/2393] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:48.733801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bcf34-d860-4c91-a8b7-9708d25a5515 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:48.745129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bcf34-d860-4c91-a8b7-9708d25a5515 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:48.749950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:48.754292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:48.754292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:48.754292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:48.754292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:48.754292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:48.754292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:48.766881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1200/2394] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:53:48.771273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8bb66-4a5d-4fe2-b56e-14c22ebf372e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:48.780593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8bb66-4a5d-4fe2-b56e-14c22ebf372e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:48.785303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:48.785947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:48.786813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:48.787110 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:48.787807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:48.789739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:48.842250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:48.924193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1195/2395] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:48 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:48.929860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:48.947551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fa3c4-b009-4676-b313-854df96f3d12 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.036011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.039485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.040330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.041090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.042018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.044873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.204098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1201/2396] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:48 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:53:49.206878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.207025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.209260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:49.222250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe07ce-78a0-423f-b649-92cd9c0ee316 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.224765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4e55a-74f4-4f13-bff1-486ff87aa841 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.231791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.232102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.232736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.232896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.233373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.233498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.233593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.233671 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] PATH_INFO: `/v3/projects/5bd5e8591859498daad7b79584dfbc0a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.234169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.235371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd5e8591859498daad7b79584dfbc0a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.236070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.237840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.251838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1202/2397] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:49.256186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.266749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb545b-53a3-4b08-b6bf-0f8ad8dbe9c8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.272730 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.273451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.273747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.274026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.275155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.276575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.292991 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1203/2398] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:49.295762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.307086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b11dd3-1a74-4f35-83e4-e0e28c82ce32 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.312354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbd54d-347b-42ca-a9bb-8f9af18adc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.313319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbd54d-347b-42ca-a9bb-8f9af18adc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.313608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbd54d-347b-42ca-a9bb-8f9af18adc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.313888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbd54d-347b-42ca-a9bb-8f9af18adc27 admin admin] PATH_INFO: `/v3/projects/b938e3b2a5d34849ab2b32d10e102e75/users/80029f3c28fb4559a64aa041bd734e4e/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.332415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbd54d-347b-42ca-a9bb-8f9af18adc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b938e3b2a5d34849ab2b32d10e102e75, user_id=80029f3c28fb4559a64aa041bd734e4e, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.336168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbd54d-347b-42ca-a9bb-8f9af18adc27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.446230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1204/2399] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:49 2022] PUT /identity/v3/projects/b938e3b2a5d34849ab2b32d10e102e75/users/80029f3c28fb4559a64aa041bd734e4e/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:49.457031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce428b-3f6a-4b75-bb0f-fd89b5c4344b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.457031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce428b-3f6a-4b75-bb0f-fd89b5c4344b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.457031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce428b-3f6a-4b75-bb0f-fd89b5c4344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.460525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd5e8591859498daad7b79584dfbc0a', 'request_id': 'req-9df696bc-4acc-4b54-bddd-cab9243f7086'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:49.468159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd5e8591859498daad7b79584dfbc0a', 'request_id': 'req-9df696bc-4acc-4b54-bddd-cab9243f7086'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:49.486374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-abce428b-3f6a-4b75-bb0f-fd89b5c4344b None None] MFA Rules not processed for user `80029f3c28fb4559a64aa041bd734e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:49.487605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-abce428b-3f6a-4b75-bb0f-fd89b5c4344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:49.555122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1205/2400] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:49.556865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd5e8591859498daad7b79584dfbc0a', 'request_id': 'req-9df696bc-4acc-4b54-bddd-cab9243f7086'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:49.559666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce428b-3f6a-4b75-bb0f-fd89b5c4344b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.572078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] The token cache is being invalidate because project 5bd5e8591859498daad7b79584dfbc0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:49.572418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:49.573122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9df696bc-4acc-4b54-bddd-cab9243f7086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:49.574892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1196/2401] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:49 2022] DELETE /identity/v3/projects/5bd5e8591859498daad7b79584dfbc0a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:49.581072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd8703-6127-4d69-bf34-0b8a7744532f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.581373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd8703-6127-4d69-bf34-0b8a7744532f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.581716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd8703-6127-4d69-bf34-0b8a7744532f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.607041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f9fd8703-6127-4d69-bf34-0b8a7744532f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:49.608293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9fd8703-6127-4d69-bf34-0b8a7744532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:49.642191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce428b-3f6a-4b75-bb0f-fd89b5c4344b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.647308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.647984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.648277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.648553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.649300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.653799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.679251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1197/2402] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:49.687563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e2153-d01c-429e-88c4-628efc8a9f10 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.698281 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e2153-d01c-429e-88c4-628efc8a9f10 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.698281 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e2153-d01c-429e-88c4-628efc8a9f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.706511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-279e2153-d01c-429e-88c4-628efc8a9f10 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:49.707833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-279e2153-d01c-429e-88c4-628efc8a9f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:49.717055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1198/2403] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:49.721548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e2153-d01c-429e-88c4-628efc8a9f10 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.732210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e2153-d01c-429e-88c4-628efc8a9f10 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.737268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.749971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.749971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.749971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.749971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:49.749971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.749971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.754633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1199/2404] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:53:49.760077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.770060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78c26-ad7c-4fe8-891e-68b2b9f157a3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.775758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.776411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.776695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.777000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.777756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.779710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.833617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1206/2405] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:49 2022] POST /identity/v3/projects => generated 371 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:49.841673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.841673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:49.905627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad0c74-4c3f-4c2b-8848-be78142eef3d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.910659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.912092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.912381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.912662 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.913405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.915412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.044137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:50.051235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1200/2406] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:49 2022] POST /identity/v3/projects => generated 385 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:50.055778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.118944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1066ab0-9162-4a73-be93-3dcb30e363e7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.123871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.124545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.124880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.125142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.125868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.127805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.188971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1207/2407] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:49 2022] POST /identity/v3/users => generated 292 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:50.191432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.202226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9daa43-342f-4fda-a5ba-8664663e4d89 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.207770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.209034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.209326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.209601 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.210329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.226141 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.248307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1208/2408] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:50.249591 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.260614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1201/2409] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:50 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:50.264924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.265088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6513f-67dd-4304-b955-9fa4775899df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.270307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.271846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.272206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.272553 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] PATH_INFO: `/v3/projects/33f19b3947bb4070bf2bbeb3edeb36e5/users/cfab7f831278439b84fc4a8830cd316e/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.276380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ae2d8-cc51-449c-b80b-3e9b255c0178 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.283141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.284130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.284753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.285159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.285987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.287999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.292202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f19b3947bb4070bf2bbeb3edeb36e5, user_id=cfab7f831278439b84fc4a8830cd316e, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.295027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.300188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1202/2410] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:50.303939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.318259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f5045-129a-4a63-b1fb-8beb3ff70e89 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.323109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.324163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.324163 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.324389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.325083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.326676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.339225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1203/2411] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:50.343736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.355987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552ed2c-2509-4895-9c91-641ec610cc94 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.361295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b40c3-8e09-489d-9083-5ada29eac22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.362551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b40c3-8e09-489d-9083-5ada29eac22a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.362851 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b40c3-8e09-489d-9083-5ada29eac22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.363214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b40c3-8e09-489d-9083-5ada29eac22a admin admin] PATH_INFO: `/v3/projects/b7815e6973aa406f9db9d81432e729c4/users/aa00826d0e404501951bd7e1b07e7b07/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.390629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b40c3-8e09-489d-9083-5ada29eac22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7815e6973aa406f9db9d81432e729c4, user_id=aa00826d0e404501951bd7e1b07e7b07, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.391704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1209/2412] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:50 2022] PUT /identity/v3/projects/33f19b3947bb4070bf2bbeb3edeb36e5/users/cfab7f831278439b84fc4a8830cd316e/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:50.394765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b40c3-8e09-489d-9083-5ada29eac22a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.404127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.482569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89e07e-7435-4e77-ab90-4eea3e27cdf1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.487941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.488671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.489024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.490010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.490806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.494327 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.504171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1204/2413] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:50 2022] PUT /identity/v3/projects/b7815e6973aa406f9db9d81432e729c4/users/aa00826d0e404501951bd7e1b07e7b07/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:50.516389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.516686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.517064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.518895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1210/2414] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:53:50.524919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.534161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:50.540860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536924-b139-4bda-be00-3b3cb0fcd7fa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.554836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] MFA Rules not processed for user `aa00826d0e404501951bd7e1b07e7b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:50.557031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:50.611527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7af7f-aa47-440c-aef4-3a44fea1459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.612740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7af7f-aa47-440c-aef4-3a44fea1459e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.613201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7af7f-aa47-440c-aef4-3a44fea1459e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.613526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7af7f-aa47-440c-aef4-3a44fea1459e admin admin] PATH_INFO: `/v3/projects/33f19b3947bb4070bf2bbeb3edeb36e5/users/cfab7f831278439b84fc4a8830cd316e/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.616399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7af7f-aa47-440c-aef4-3a44fea1459e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f19b3947bb4070bf2bbeb3edeb36e5, user_id=cfab7f831278439b84fc4a8830cd316e, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.619525 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7af7f-aa47-440c-aef4-3a44fea1459e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.633835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1211/2415] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:50 2022] PUT /identity/v3/projects/33f19b3947bb4070bf2bbeb3edeb36e5/users/cfab7f831278439b84fc4a8830cd316e/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:50.648041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.648337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.648616 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.661990 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1205/2416] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:50.679700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.680963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] MFA Rules not processed for user `cfab7f831278439b84fc4a8830cd316e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:50.681355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:50.682281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:50.748948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176eb9d-929a-4ae9-99de-a9ed347aa743 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.759707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1212/2417] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:53:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:50.808937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.809540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.809820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.810098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.869893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.871448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.881536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1206/2418] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:53:50.891854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:50.893785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:50.908363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06589af7-5bec-4afb-aacd-7b4d280bd27f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:50.913549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929ea80-1e7f-4701-a654-9a08c378d667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:50.914646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929ea80-1e7f-4701-a654-9a08c378d667 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:50.920205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929ea80-1e7f-4701-a654-9a08c378d667 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:50.920205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929ea80-1e7f-4701-a654-9a08c378d667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:50.930185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929ea80-1e7f-4701-a654-9a08c378d667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:50.931828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929ea80-1e7f-4701-a654-9a08c378d667 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:50.948526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1213/2419] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:51.704288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:51.720581 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0255f-ad60-49fb-90bb-a6914b20fe0d service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:51.739008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6af93-4c98-4e2f-82e1-0e2d9afbbb5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:51.740386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6af93-4c98-4e2f-82e1-0e2d9afbbb5a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:51.741682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6af93-4c98-4e2f-82e1-0e2d9afbbb5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:51.741682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6af93-4c98-4e2f-82e1-0e2d9afbbb5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:51.746225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a4c6af93-4c98-4e2f-82e1-0e2d9afbbb5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:51.791433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:51.840481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6af93-4c98-4e2f-82e1-0e2d9afbbb5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:51.842916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6af93-4c98-4e2f-82e1-0e2d9afbbb5a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:51.882774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1207/2420] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:52.322240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:52.910498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:53.025148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:53.794536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252d3df-62ed-4aac-a886-fd4d1fad2e11 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:53.795243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252d3df-62ed-4aac-a886-fd4d1fad2e11 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:53.795243 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252d3df-62ed-4aac-a886-fd4d1fad2e11 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:53.796484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1214/2421] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:53:53.822894 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87699d76-5b8e-438e-a74f-81c0e39dc20d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:53.826840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87699d76-5b8e-438e-a74f-81c0e39dc20d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:53.826840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87699d76-5b8e-438e-a74f-81c0e39dc20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:53.849096 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-87699d76-5b8e-438e-a74f-81c0e39dc20d None None] MFA Rules not processed for user `ec899ae414744dde9ec9b76aac1ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:53.850399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87699d76-5b8e-438e-a74f-81c0e39dc20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:53.946077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1208/2422] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:53:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:53.965153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252d3df-62ed-4aac-a886-fd4d1fad2e11 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:53.967104 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b252d3df-62ed-4aac-a886-fd4d1fad2e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:54.047922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252d3df-62ed-4aac-a886-fd4d1fad2e11 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:54.052288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94dcf5-896c-4988-96f9-8f51e5b47015 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:54.052988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94dcf5-896c-4988-96f9-8f51e5b47015 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:54.053279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94dcf5-896c-4988-96f9-8f51e5b47015 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:54.053559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94dcf5-896c-4988-96f9-8f51e5b47015 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:54.065272 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94dcf5-896c-4988-96f9-8f51e5b47015 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:54.066791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94dcf5-896c-4988-96f9-8f51e5b47015 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:54.078875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1215/2423] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:53:54.552694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f5274-8c67-44d5-adb3-d0c597c77608 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:54.552694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f5274-8c67-44d5-adb3-d0c597c77608 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:54.552694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f5274-8c67-44d5-adb3-d0c597c77608 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:54.553767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1209/2424] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 18 10:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:53:54.562532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc3021-43dd-4d03-b322-58294a00a1fa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:54.562998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc3021-43dd-4d03-b322-58294a00a1fa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:54.563519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc3021-43dd-4d03-b322-58294a00a1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:54.586379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-53cc3021-43dd-4d03-b322-58294a00a1fa None None] MFA Rules not processed for user `a48b0fdacb2c43649916304e1237f39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:54.586379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53cc3021-43dd-4d03-b322-58294a00a1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:54.642356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1216/2425] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 18 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:54.657048 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f5274-8c67-44d5-adb3-d0c597c77608 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:54.658844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b0f5274-8c67-44d5-adb3-d0c597c77608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:54.673151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f5274-8c67-44d5-adb3-d0c597c77608 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:54.678326 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:54.682064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:54.682064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:54.682064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:54.687933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:54.689431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:54.700322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1210/2426] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:59.146684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb0ab-b151-4153-91ff-793e243f6406 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:59.147209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb0ab-b151-4153-91ff-793e243f6406 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:59.147209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb0ab-b151-4153-91ff-793e243f6406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:59.169278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fc0cb0ab-b151-4153-91ff-793e243f6406 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:59.169278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc0cb0ab-b151-4153-91ff-793e243f6406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:59.187911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:59.242071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1217/2427] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:59.255963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:59.257598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:59.331803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3c890-71c4-4182-b5e0-43e1dda423d6 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:59.337559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6c15d-df1e-4768-a0b3-a0829318c78d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:59.338215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c15d-df1e-4768-a0b3-a0829318c78d service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:59.338545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c15d-df1e-4768-a0b3-a0829318c78d service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:59.338825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c15d-df1e-4768-a0b3-a0829318c78d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:59.347596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6c15d-df1e-4768-a0b3-a0829318c78d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:59.349372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6c15d-df1e-4768-a0b3-a0829318c78d service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:59.357651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1211/2428] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:53:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:54:04.711630 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cb0ab-b151-4153-91ff-793e243f6406 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:04.878624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cb0ab-b151-4153-91ff-793e243f6406 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:04.889873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:04.891354 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:04.891816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:04.892242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] PATH_INFO: `/v3/users/aa00826d0e404501951bd7e1b07e7b07` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:04.893731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa00826d0e404501951bd7e1b07e7b07)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:04.897366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:05.170848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa00826d0e404501951bd7e1b07e7b07'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:05.192869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa00826d0e404501951bd7e1b07e7b07'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:05.299194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa00826d0e404501951bd7e1b07e7b07'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:05.299900 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa00826d0e404501951bd7e1b07e7b07'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:05.300552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa00826d0e404501951bd7e1b07e7b07'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:05.334008 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1218/2429] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:04 2022] DELETE /identity/v3/users/aa00826d0e404501951bd7e1b07e7b07 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:05.339848 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:05.344860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:05.344860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:05.368808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:05.370413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:05.482778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:05.510745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1212/2430] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:05.525404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:05.532537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:05.608233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:05.612856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a709e-4912-4a1f-b30f-fd7dca1f6d05 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:05.617676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4398c2a-5df2-4767-8a54-59e5f0c32bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:05.618588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398c2a-5df2-4767-8a54-59e5f0c32bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:05.618983 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398c2a-5df2-4767-8a54-59e5f0c32bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:05.619292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398c2a-5df2-4767-8a54-59e5f0c32bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:05.633233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4398c2a-5df2-4767-8a54-59e5f0c32bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:05.635034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4398c2a-5df2-4767-8a54-59e5f0c32bf9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:05.648310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1219/2431] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:07.219054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:07.220559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:07.260868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17738ce-1672-452c-9344-faf5e97d023d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:07.287390 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:07.301708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:07.306942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:07.309756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] PATH_INFO: `/v3/projects/b7815e6973aa406f9db9d81432e729c4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:07.314944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7815e6973aa406f9db9d81432e729c4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:07.327646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:07.569276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7815e6973aa406f9db9d81432e729c4', 'request_id': 'req-0722d2d8-0507-4e3b-ad56-dd02f81739fe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:07.592132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7815e6973aa406f9db9d81432e729c4', 'request_id': 'req-0722d2d8-0507-4e3b-ad56-dd02f81739fe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:07.670260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7815e6973aa406f9db9d81432e729c4', 'request_id': 'req-0722d2d8-0507-4e3b-ad56-dd02f81739fe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:07.693537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] The token cache is being invalidate because project b7815e6973aa406f9db9d81432e729c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:07.694452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:07.694870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0722d2d8-0507-4e3b-ad56-dd02f81739fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:07.705525 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1213/2432] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:07 2022] DELETE /identity/v3/projects/b7815e6973aa406f9db9d81432e729c4 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:07.713076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b6d7-2d3d-4c0c-8e8b-26d0ce68b3b2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:07.714042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b6d7-2d3d-4c0c-8e8b-26d0ce68b3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:07.714042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b6d7-2d3d-4c0c-8e8b-26d0ce68b3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:07.734980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9044b6d7-2d3d-4c0c-8e8b-26d0ce68b3b2 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:07.736274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9044b6d7-2d3d-4c0c-8e8b-26d0ce68b3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:07.825565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1220/2433] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:07.834470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fabcea-fb73-4d74-918e-49634f131324 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:07.834937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fabcea-fb73-4d74-918e-49634f131324 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:07.835375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fabcea-fb73-4d74-918e-49634f131324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:07.891843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-27fabcea-fb73-4d74-918e-49634f131324 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:07.894160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27fabcea-fb73-4d74-918e-49634f131324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:07.908466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1214/2434] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:54:07.913856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044b6d7-2d3d-4c0c-8e8b-26d0ce68b3b2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:07.930823 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044b6d7-2d3d-4c0c-8e8b-26d0ce68b3b2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:07.937153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:07.938001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:07.938406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:07.938821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:07.939614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:54:07.939961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:07.941748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:07.960052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1221/2435] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:54:07.966116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fabcea-fb73-4d74-918e-49634f131324 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:07.982108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fabcea-fb73-4d74-918e-49634f131324 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:07.988407 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:07.989822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:07.990257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:07.990593 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:07.991580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:07.994172 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:08.124447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1215/2436] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:54:07 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:08.130249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:08.142303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b60a34-6398-4a85-b298-e4d774c7b67a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:08.233754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:08.234832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:08.235189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:08.235474 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:08.236424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:08.239064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:08.392287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1222/2437] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:54:08 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:08.397622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:08.411183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b8a82-68aa-41b3-adac-4c2c186f1bec admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:08.415473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:08.416270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:08.416688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:08.417433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:08.418082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:08.419944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:08.438492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1216/2438] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:08.442242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:08.459217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ac984-8741-438f-b834-3082bc0102f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:08.465746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed19459-df7c-45bb-b21a-414c983ce762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:08.473361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed19459-df7c-45bb-b21a-414c983ce762 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:08.473361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed19459-df7c-45bb-b21a-414c983ce762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:08.473361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed19459-df7c-45bb-b21a-414c983ce762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:08.473361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed19459-df7c-45bb-b21a-414c983ce762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:08.473361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed19459-df7c-45bb-b21a-414c983ce762 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:08.492514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1223/2439] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:08.496051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:08.508540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e3c9a-2ef8-465f-852b-9c76fa165f1d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:08.516090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:08.516090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:08.516090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:08.516090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] PATH_INFO: `/v3/projects/c1d70e3c660540c99b8236e3908be637/users/957b29c028d44ada995c8e0e8cdc9ab7/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:08.540445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d70e3c660540c99b8236e3908be637, user_id=957b29c028d44ada995c8e0e8cdc9ab7, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:08.548309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:08.668422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1217/2440] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:54:08 2022] PUT /identity/v3/projects/c1d70e3c660540c99b8236e3908be637/users/957b29c028d44ada995c8e0e8cdc9ab7/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:08.684873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6d48-23fb-4fef-be68-20b9ce8effcf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:08.685202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6d48-23fb-4fef-be68-20b9ce8effcf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:08.685439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6d48-23fb-4fef-be68-20b9ce8effcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:08.757873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dc6e6d48-23fb-4fef-be68-20b9ce8effcf None None] MFA Rules not processed for user `957b29c028d44ada995c8e0e8cdc9ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:08.763270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dc6e6d48-23fb-4fef-be68-20b9ce8effcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:08.967244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1224/2441] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:08.999436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:08.999436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:09.091860 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d746-6c7d-4dfa-a50d-81e391753234 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:09.107688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d259f53-0924-4021-ade8-a82c1acd778a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:09.107688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d259f53-0924-4021-ade8-a82c1acd778a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:09.107688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d259f53-0924-4021-ade8-a82c1acd778a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:09.107688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d259f53-0924-4021-ade8-a82c1acd778a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:09.121263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d259f53-0924-4021-ade8-a82c1acd778a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:09.122773 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d259f53-0924-4021-ade8-a82c1acd778a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:09.136325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1218/2442] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:54:09.148639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:12.858364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e6d48-23fb-4fef-be68-20b9ce8effcf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:12.941343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e6d48-23fb-4fef-be68-20b9ce8effcf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:12.948186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:12.949262 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:12.949548 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:12.949827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] PATH_INFO: `/v3/users/957b29c028d44ada995c8e0e8cdc9ab7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:12.951109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=957b29c028d44ada995c8e0e8cdc9ab7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:12.953152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:13.267728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957b29c028d44ada995c8e0e8cdc9ab7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:13.281039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957b29c028d44ada995c8e0e8cdc9ab7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:13.406835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957b29c028d44ada995c8e0e8cdc9ab7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:13.407479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957b29c028d44ada995c8e0e8cdc9ab7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:13.408109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957b29c028d44ada995c8e0e8cdc9ab7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:13.422993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1225/2443] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:12 2022] DELETE /identity/v3/users/957b29c028d44ada995c8e0e8cdc9ab7 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:13.427774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:13.428710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:13.428710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:13.448153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:13.457935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:13.460834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:13.528594 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1219/2444] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:13.541881 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:13.549240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:13.620442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75af38-4066-4ee2-a80d-4eb7fa4ddb6a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:13.624275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5ba51-b8ba-443c-9304-45092d4589ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:13.624969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5ba51-b8ba-443c-9304-45092d4589ed service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:13.625260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5ba51-b8ba-443c-9304-45092d4589ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:13.625539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5ba51-b8ba-443c-9304-45092d4589ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:13.637423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5ba51-b8ba-443c-9304-45092d4589ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:13.639088 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5ba51-b8ba-443c-9304-45092d4589ed service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:13.652704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1226/2445] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:14.265808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:14.270460 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:14.279545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b94f59-f3d9-42dc-b74c-2a3147865ee4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:14.284532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:14.285980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:14.286257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:14.286540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] PATH_INFO: `/v3/projects/c1d70e3c660540c99b8236e3908be637` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:14.287745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d70e3c660540c99b8236e3908be637)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:14.289718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:14.432192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d70e3c660540c99b8236e3908be637', 'request_id': 'req-fa7ee531-d067-41a2-8d81-35771b3954d3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:14.438473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d70e3c660540c99b8236e3908be637', 'request_id': 'req-fa7ee531-d067-41a2-8d81-35771b3954d3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:14.561444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d70e3c660540c99b8236e3908be637', 'request_id': 'req-fa7ee531-d067-41a2-8d81-35771b3954d3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:14.574981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] The token cache is being invalidate because project c1d70e3c660540c99b8236e3908be637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:14.575275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:14.575932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:14.577522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1220/2446] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:14 2022] DELETE /identity/v3/projects/c1d70e3c660540c99b8236e3908be637 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:16.799985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:16.800483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:16.801288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:16.822615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:16.824817 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:16.829520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:16.898209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1227/2447] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 18 10:54:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:16.916646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:16.916646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:17.000190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ee531-d067-41a2-8d81-35771b3954d3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:17.005965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:17.006623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:17.006907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:17.007187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:17.015712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:17.017428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:17.026616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1221/2448] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:54:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:54:27.023059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:27.041499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=21 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:35.803500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:35.804871 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:35.866110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd22a0-6b96-4dcb-bca1-695ab5f3d62f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:35.869985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:35.870632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:35.870912 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:35.871185 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:35.872981 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:35.943681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:35.945203 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:35.955740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1228/2449] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:40.013640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:40.015380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:40.031566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5a9ba-bbe4-4af3-97c8-f18fb07887f6 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:40.036701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:40.037632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:40.037910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:40.038186 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] PATH_INFO: `/v3/users/3954fd4abf6b4c3a82280c762292a3e7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:40.039449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3954fd4abf6b4c3a82280c762292a3e7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:40.041783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:40.347985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3954fd4abf6b4c3a82280c762292a3e7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:40.355379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3954fd4abf6b4c3a82280c762292a3e7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:40.368874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3954fd4abf6b4c3a82280c762292a3e7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:40.469499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3954fd4abf6b4c3a82280c762292a3e7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:40.470158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec19a186-80c2-4ef9-8ad1-4b50f17c797a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3954fd4abf6b4c3a82280c762292a3e7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:40.480697 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1222/2450] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:40 2022] DELETE /identity/v3/users/3954fd4abf6b4c3a82280c762292a3e7 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:40.779965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:40.791162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48681db8-9d26-4b0f-bc5c-9beab1d8320d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:40.854575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:40.855283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:40.855647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:40.856027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] PATH_INFO: `/v3/projects/04716d82e1084ec58b2a4909710a6ce8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:40.857424 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] RBAC: Authorizing `identity:delete_project(project_id=04716d82e1084ec58b2a4909710a6ce8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:40.864768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:41.003423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04716d82e1084ec58b2a4909710a6ce8', 'request_id': 'req-c6f23012-ab71-41ac-b456-c19a4e2d321a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:41.006097 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:41.218618 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04716d82e1084ec58b2a4909710a6ce8', 'request_id': 'req-c6f23012-ab71-41ac-b456-c19a4e2d321a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:41.225686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04716d82e1084ec58b2a4909710a6ce8', 'request_id': 'req-c6f23012-ab71-41ac-b456-c19a4e2d321a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:41.240808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] The token cache is being invalidate because project 04716d82e1084ec58b2a4909710a6ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:41.240808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:41.240808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f23012-ab71-41ac-b456-c19a4e2d321a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:41.240808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1229/2451] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:40 2022] DELETE /identity/v3/projects/04716d82e1084ec58b2a4909710a6ce8 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:41.244534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138da67-880e-4004-ba1e-f321b7a11460 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.245190 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138da67-880e-4004-ba1e-f321b7a11460 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.245354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138da67-880e-4004-ba1e-f321b7a11460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.263715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8138da67-880e-4004-ba1e-f321b7a11460 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:41.264967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8138da67-880e-4004-ba1e-f321b7a11460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:41.330159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1223/2452] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:41.339635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59610fd-74b3-4b20-9dff-0ad38603489d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.339962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59610fd-74b3-4b20-9dff-0ad38603489d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.340245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59610fd-74b3-4b20-9dff-0ad38603489d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.360506 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f59610fd-74b3-4b20-9dff-0ad38603489d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:41.366909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f59610fd-74b3-4b20-9dff-0ad38603489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:41.370920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1230/2453] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:41.375408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138da67-880e-4004-ba1e-f321b7a11460 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:41.389854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138da67-880e-4004-ba1e-f321b7a11460 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:41.395703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:41.396388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.396673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.396989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.397745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:54:41.398107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:41.399728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:41.412823 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1224/2454] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:54:41.418077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59610fd-74b3-4b20-9dff-0ad38603489d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:41.430080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59610fd-74b3-4b20-9dff-0ad38603489d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:41.432938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:41.433603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.433913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.434189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.434894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:41.436922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:41.602302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1231/2455] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:54:41 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:41.603651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:41.615013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8497c0-3944-42e0-9f2b-f9f6f7199c3a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:41.679516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:41.680120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.680411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.680807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.681481 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:41.683443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:41.690271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:41.837006 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1225/2456] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:54:41 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:41.841722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:41.853861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255955ac-7438-41f3-a374-eea2d85dce69 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:41.859439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:41.863982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.863982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.863982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.863982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:41.864469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:41.876364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1232/2457] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:41.881065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:41.892891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4775c6-6124-4336-835a-81309f14b154 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:41.898277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75a61e-31f5-4ebf-8cee-7b466565fdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:41.899057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75a61e-31f5-4ebf-8cee-7b466565fdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.899273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75a61e-31f5-4ebf-8cee-7b466565fdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.899555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75a61e-31f5-4ebf-8cee-7b466565fdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.900371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75a61e-31f5-4ebf-8cee-7b466565fdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:41.902151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75a61e-31f5-4ebf-8cee-7b466565fdb3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:41.915341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1226/2458] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:41.919887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:41.931439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9678f5-64ab-4606-8829-3478bd26a783 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:41.937486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:41.938430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:41.938709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:41.938985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] PATH_INFO: `/v3/projects/9cd3cbe5f4c4466cac183fb444cce626/users/6085a183c88b488e807c96d14be849f6/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:41.957496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd3cbe5f4c4466cac183fb444cce626, user_id=6085a183c88b488e807c96d14be849f6, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:41.960316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:42.070852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1233/2459] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:54:41 2022] PUT /identity/v3/projects/9cd3cbe5f4c4466cac183fb444cce626/users/6085a183c88b488e807c96d14be849f6/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:42.089838 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c2f4b-1466-444c-a746-ab54218bcd63 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:42.090131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c2f4b-1466-444c-a746-ab54218bcd63 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:42.090412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c2f4b-1466-444c-a746-ab54218bcd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:42.120441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ee4c2f4b-1466-444c-a746-ab54218bcd63 None None] MFA Rules not processed for user `6085a183c88b488e807c96d14be849f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:42.121752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee4c2f4b-1466-444c-a746-ab54218bcd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:42.191235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1227/2460] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:42.202835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:42.207417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:42.285265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d608b2-3e41-460f-a508-97cfde7ef2e4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:42.291850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155eb4b-e1c9-4420-86ac-778fb2fddfd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:42.292499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155eb4b-e1c9-4420-86ac-778fb2fddfd3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:42.292802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155eb4b-e1c9-4420-86ac-778fb2fddfd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:42.293125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155eb4b-e1c9-4420-86ac-778fb2fddfd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:42.302911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155eb4b-e1c9-4420-86ac-778fb2fddfd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:42.304550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155eb4b-e1c9-4420-86ac-778fb2fddfd3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:42.317666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1234/2461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:48.008471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c2f4b-1466-444c-a746-ab54218bcd63 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:48.080492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c2f4b-1466-444c-a746-ab54218bcd63 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:48.085727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:48.086635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:48.086915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:48.087202 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] PATH_INFO: `/v3/users/6085a183c88b488e807c96d14be849f6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:48.088412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6085a183c88b488e807c96d14be849f6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:48.090677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:48.375924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6085a183c88b488e807c96d14be849f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:48.383692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6085a183c88b488e807c96d14be849f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:48.396156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:48.398491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6085a183c88b488e807c96d14be849f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:48.509340 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6085a183c88b488e807c96d14be849f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:48.510149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6085a183c88b488e807c96d14be849f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:48.516615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1228/2462] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:48 2022] DELETE /identity/v3/users/6085a183c88b488e807c96d14be849f6 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:48.521435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:48.521931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:48.522032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:48.545833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:48.547041 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:48.610787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1235/2463] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:48.621710 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:48.623167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:48.681473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013624ad-3202-4929-8353-7aa43c070ee0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:48.685072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509f4f1-3d33-4c6f-ac3c-e869411f5ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:48.685736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f4f1-3d33-4c6f-ac3c-e869411f5ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:48.686005 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f4f1-3d33-4c6f-ac3c-e869411f5ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:48.686334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f4f1-3d33-4c6f-ac3c-e869411f5ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:48.695598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509f4f1-3d33-4c6f-ac3c-e869411f5ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:48.697073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509f4f1-3d33-4c6f-ac3c-e869411f5ae5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:48.706721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1229/2464] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:54:49.191573 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:49.193080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:49.200479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:49.206690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c893a-d98a-447a-9ef5-8ce233ba5296 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:49.211681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:49.212620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:49.212910 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:49.213184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] PATH_INFO: `/v3/projects/9cd3cbe5f4c4466cac183fb444cce626` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:49.214645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd3cbe5f4c4466cac183fb444cce626)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:49.216636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:49.361722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd3cbe5f4c4466cac183fb444cce626', 'request_id': 'req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:49.469232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd3cbe5f4c4466cac183fb444cce626', 'request_id': 'req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:49.476719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd3cbe5f4c4466cac183fb444cce626', 'request_id': 'req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:49.487645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] The token cache is being invalidate because project 9cd3cbe5f4c4466cac183fb444cce626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:49.487997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:49.488592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6b5c1184-6c9f-48bb-b29f-3ee8b25fadb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:49.490188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1236/2465] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:49 2022] DELETE /identity/v3/projects/9cd3cbe5f4c4466cac183fb444cce626 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:49.495427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9d6a-9df9-4920-9b60-ea2a75f9c94a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:49.495916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9d6a-9df9-4920-9b60-ea2a75f9c94a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:49.495916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9d6a-9df9-4920-9b60-ea2a75f9c94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:49.514947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e46b9d6a-9df9-4920-9b60-ea2a75f9c94a None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:49.516145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e46b9d6a-9df9-4920-9b60-ea2a75f9c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:49.582452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1230/2466] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:49.593696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68ad8-c7de-44e8-8675-467f2dad1ca5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:49.594212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68ad8-c7de-44e8-8675-467f2dad1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:49.594408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68ad8-c7de-44e8-8675-467f2dad1ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:49.615177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9cf68ad8-c7de-44e8-8675-467f2dad1ca5 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:49.616425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9cf68ad8-c7de-44e8-8675-467f2dad1ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:49.627367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1237/2467] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:49.631214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b9d6a-9df9-4920-9b60-ea2a75f9c94a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:49.642029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b9d6a-9df9-4920-9b60-ea2a75f9c94a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:49.646941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:49.647818 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:49.647938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:49.648169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:49.648964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:54:49.649287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:49.651021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:49.665486 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1231/2468] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:54:49.670027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf68ad8-c7de-44e8-8675-467f2dad1ca5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:49.680565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf68ad8-c7de-44e8-8675-467f2dad1ca5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:49.685499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:49.686212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:49.686499 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:49.686779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:49.687489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:49.689543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:49.792532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1238/2469] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:54:49 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:49.796863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:49.810691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d662ff-db43-4a4b-9622-b9e7fa0344a9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:49.868984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:49.869665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:49.869950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:49.870229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:49.870925 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:49.872873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:50.042521 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1232/2470] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:54:49 2022] POST /identity/v3/users => generated 284 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:54:50.046840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:50.058406 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c305e-e47b-4898-ad6b-98a4a6697b31 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:50.063962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:50.064632 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:50.064967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:50.065793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:50.066537 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:50.068161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:50.079460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1239/2471] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:50.084542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:50.095528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c967df3-5db9-4b34-a726-379dc7e75b7c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:50.100557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc44f2-c924-438c-a50d-9e6cfd8261ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:50.101244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc44f2-c924-438c-a50d-9e6cfd8261ee admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:50.101527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc44f2-c924-438c-a50d-9e6cfd8261ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:50.101874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc44f2-c924-438c-a50d-9e6cfd8261ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:50.102599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc44f2-c924-438c-a50d-9e6cfd8261ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:50.104234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc44f2-c924-438c-a50d-9e6cfd8261ee admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:50.105635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:50.116396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1233/2472] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:50.120631 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:50.131668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74b70-26ab-4a26-bf7c-935d1115f7f8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:50.136535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:50.137455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:50.137741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:50.138018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] PATH_INFO: `/v3/projects/0b2a8b6cbcf84cdf8ecdba5de18eff7f/users/efd75526a1a44cb58329f4548c37af4f/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:50.153469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2a8b6cbcf84cdf8ecdba5de18eff7f, user_id=efd75526a1a44cb58329f4548c37af4f, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:50.156147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:50.276485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1240/2473] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:54:50 2022] PUT /identity/v3/projects/0b2a8b6cbcf84cdf8ecdba5de18eff7f/users/efd75526a1a44cb58329f4548c37af4f/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:54:50.288483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:50.288805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:50.289087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:50.317926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] MFA Rules not processed for user `efd75526a1a44cb58329f4548c37af4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:50.319245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:50.383376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1234/2474] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:50.396188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:50.397902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:50.454880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b42243-ee35-4853-92ac-b4c4f4c0a1df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:50.460260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d41631-0226-4552-b6ed-d4741b7bdd46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:50.460963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d41631-0226-4552-b6ed-d4741b7bdd46 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:50.461254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d41631-0226-4552-b6ed-d4741b7bdd46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:50.461532 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d41631-0226-4552-b6ed-d4741b7bdd46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:50.470466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d41631-0226-4552-b6ed-d4741b7bdd46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:50.471977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d41631-0226-4552-b6ed-d4741b7bdd46 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:50.480944 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1241/2475] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:51.420067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:51.421498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:51.496973 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a351d75-a462-4a87-8f4c-b6b2f4bc2118 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:51.501523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:51.502719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:51.502719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:51.503511 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] PATH_INFO: `/v3/users/80029f3c28fb4559a64aa041bd734e4e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:51.504384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=80029f3c28fb4559a64aa041bd734e4e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:51.508980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:51.791717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80029f3c28fb4559a64aa041bd734e4e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:51.799459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80029f3c28fb4559a64aa041bd734e4e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:51.816107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80029f3c28fb4559a64aa041bd734e4e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:51.983604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80029f3c28fb4559a64aa041bd734e4e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:51.984262 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80029f3c28fb4559a64aa041bd734e4e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:51.991478 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1235/2476] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:51 2022] DELETE /identity/v3/users/80029f3c28fb4559a64aa041bd734e4e => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:51.996029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:51.997720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:51.997720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:52.013911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:52.015126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:52.035745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:52.075873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1242/2477] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:52.086474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:52.087900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:52.094836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:52.146948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c9d4f-0344-4357-a578-504a6c2e88dc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:52.150708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:52.151425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:52.151713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:52.152002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:52.162065 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:52.163510 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:52.173413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1236/2478] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:52.711570 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:52.713613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:52.728120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53bcb5-26b9-4f62-ad30-9cd44d6f7fde None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:52.733756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:52.734641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:52.734937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:52.735216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] PATH_INFO: `/v3/projects/b938e3b2a5d34849ab2b32d10e102e75` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:52.736459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b938e3b2a5d34849ab2b32d10e102e75)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:52.744543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:52.865543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b938e3b2a5d34849ab2b32d10e102e75', 'request_id': 'req-15d5c795-9b4b-486f-a984-7f08b6807ce5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.000692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b938e3b2a5d34849ab2b32d10e102e75', 'request_id': 'req-15d5c795-9b4b-486f-a984-7f08b6807ce5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.008899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b938e3b2a5d34849ab2b32d10e102e75', 'request_id': 'req-15d5c795-9b4b-486f-a984-7f08b6807ce5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.019795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] The token cache is being invalidate because project b938e3b2a5d34849ab2b32d10e102e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:53.020091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.020846 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.022607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1243/2479] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:52 2022] DELETE /identity/v3/projects/b938e3b2a5d34849ab2b32d10e102e75 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:53.026556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:53.037028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b040e81-053f-4b30-b46a-6de5721beacc service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:53.096883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:53.097765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:53.098051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:53.098327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] PATH_INFO: `/v3/users/cfab7f831278439b84fc4a8830cd316e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:53.099514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfab7f831278439b84fc4a8830cd316e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:53.101522 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:53.475796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfab7f831278439b84fc4a8830cd316e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.483435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfab7f831278439b84fc4a8830cd316e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.496151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfab7f831278439b84fc4a8830cd316e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.652029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfab7f831278439b84fc4a8830cd316e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.652651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26dc5520-039e-453b-a4d0-fed3aeadc827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfab7f831278439b84fc4a8830cd316e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:53.660547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1237/2480] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:53 2022] DELETE /identity/v3/users/cfab7f831278439b84fc4a8830cd316e => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:54.156459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:54.158032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:54.172516 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5c795-9b4b-486f-a984-7f08b6807ce5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:54.242108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:54.242984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:54.243269 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:54.243550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] PATH_INFO: `/v3/projects/33f19b3947bb4070bf2bbeb3edeb36e5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:54.244799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33f19b3947bb4070bf2bbeb3edeb36e5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:54.246747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:54.374417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33f19b3947bb4070bf2bbeb3edeb36e5', 'request_id': 'req-e2a49f74-51e9-4850-9685-b6e67fc9bf32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:54.380438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:54.498318 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33f19b3947bb4070bf2bbeb3edeb36e5', 'request_id': 'req-e2a49f74-51e9-4850-9685-b6e67fc9bf32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:54.505848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33f19b3947bb4070bf2bbeb3edeb36e5', 'request_id': 'req-e2a49f74-51e9-4850-9685-b6e67fc9bf32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:54.518795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] The token cache is being invalidate because project 33f19b3947bb4070bf2bbeb3edeb36e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:54.519362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:54.520496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e2a49f74-51e9-4850-9685-b6e67fc9bf32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:54.522489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1244/2481] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:54 2022] DELETE /identity/v3/projects/33f19b3947bb4070bf2bbeb3edeb36e5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:54.527588 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec70b-b729-459f-86fe-e0bf4358dcd6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:54.528210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec70b-b729-459f-86fe-e0bf4358dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:54.528366 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec70b-b729-459f-86fe-e0bf4358dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:54.547421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-97eec70b-b729-459f-86fe-e0bf4358dcd6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:54.548724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-97eec70b-b729-459f-86fe-e0bf4358dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:54.571576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:54.615683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1238/2482] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:54.623921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f8e8-3db8-45a1-b443-130889edb303 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:54.624214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f8e8-3db8-45a1-b443-130889edb303 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:54.624494 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f8e8-3db8-45a1-b443-130889edb303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:54.645749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e721f8e8-3db8-45a1-b443-130889edb303 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:54.647007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e721f8e8-3db8-45a1-b443-130889edb303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:54.655324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1245/2483] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:54.669079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec70b-b729-459f-86fe-e0bf4358dcd6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:54.679465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec70b-b729-459f-86fe-e0bf4358dcd6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:54.684191 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:54.685000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:54.685291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:54.685573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:54.686328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:54:54.686638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:54.688277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:54.702601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1239/2484] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:54:54.708524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f8e8-3db8-45a1-b443-130889edb303 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:54.718669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f8e8-3db8-45a1-b443-130889edb303 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:54.725107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:54.730429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:54.730429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:54.730429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:54.730429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:54.730429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:54.936083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1246/2485] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:54:54 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:54.941071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:54.951063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9d538-624a-4c1b-a672-0f8d33b3a039 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:55.012867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:55.013549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:55.013833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:55.014136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:55.014845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:55.016756 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:55.162101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1240/2486] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:54:54 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:55.176903 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:55.187644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79872c-f1bf-4ea7-938c-b17906d2c5d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:55.193234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:55.193985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:55.194267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:55.194545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:55.195277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:55.196997 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:55.209405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1247/2487] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:55.213881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:55.224587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fda84-d526-48b5-8bf0-4f95e3354f4b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:55.230363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be5ef6-c946-4e11-9756-ab019799f980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:55.231119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be5ef6-c946-4e11-9756-ab019799f980 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:55.231413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be5ef6-c946-4e11-9756-ab019799f980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:55.231698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be5ef6-c946-4e11-9756-ab019799f980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:55.232465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be5ef6-c946-4e11-9756-ab019799f980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:55.234199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be5ef6-c946-4e11-9756-ab019799f980 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:55.245566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1241/2488] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:55.249812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:55.261057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d21f6-2ebc-4234-82bf-dff1358e6a2c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:55.265638 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:55.266518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:55.266805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:55.267111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] PATH_INFO: `/v3/projects/584fdec4218d4144915cf0be579f7353/users/56563b7e1e804dcd9cf8242d09af1b5e/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:55.286017 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=584fdec4218d4144915cf0be579f7353, user_id=56563b7e1e804dcd9cf8242d09af1b5e, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:55.312684 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:55.493747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1248/2489] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:54:55 2022] PUT /identity/v3/projects/584fdec4218d4144915cf0be579f7353/users/56563b7e1e804dcd9cf8242d09af1b5e/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:55.505496 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4efa-47f9-4f92-86df-90abfdb4b549 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:55.505801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4efa-47f9-4f92-86df-90abfdb4b549 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:55.506088 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4efa-47f9-4f92-86df-90abfdb4b549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:55.534955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eb8d4efa-47f9-4f92-86df-90abfdb4b549 None None] MFA Rules not processed for user `56563b7e1e804dcd9cf8242d09af1b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:55.537031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb8d4efa-47f9-4f92-86df-90abfdb4b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:55.606168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1242/2490] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:55.610825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:55.673519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03da71-108e-44da-92cf-9e14cd64f6aa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:55.679697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:55.680396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:55.680681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:55.681028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:55.681759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:55.683697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:55.855115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1249/2491] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:54:55 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:55.859325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d4efa-47f9-4f92-86df-90abfdb4b549 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:55.871257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d4efa-47f9-4f92-86df-90abfdb4b549 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:55.889853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:55.937064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:55.937753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:55.938040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:55.938316 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:55.939014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:55.941166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:56.104847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1243/2492] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:54:55 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:54:56.108580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:56.118963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7649ac-9533-4925-84c6-18bc83bdc854 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:56.123554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:56.128302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:56.128302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:56.128302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:56.128302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:56.128302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:56.139403 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1250/2493] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:56.144305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:56.154544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0193f26-01bd-409c-94bd-cb823e760cde admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:56.159502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:56.166520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:56.166520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:56.166520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] PATH_INFO: `/v3/projects/eb4ce2a9b2e94ca4a00b63df1fbea4e5/users/3a43b374a97f499e9cb4fa777895d038/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:56.176868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4ce2a9b2e94ca4a00b63df1fbea4e5, user_id=3a43b374a97f499e9cb4fa777895d038, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:56.179536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:56.339188 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1244/2494] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:54:56 2022] PUT /identity/v3/projects/eb4ce2a9b2e94ca4a00b63df1fbea4e5/users/3a43b374a97f499e9cb4fa777895d038/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:56.350353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:56.362825 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e581cd-a057-4276-ab7d-5a29621f99e8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:56.420216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:56.435450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fc489-5a9c-47e0-8f62-04093368105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:56.437972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fc489-5a9c-47e0-8f62-04093368105a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:56.437972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fc489-5a9c-47e0-8f62-04093368105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:56.437972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fc489-5a9c-47e0-8f62-04093368105a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:56.437972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fc489-5a9c-47e0-8f62-04093368105a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:56.439168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fc489-5a9c-47e0-8f62-04093368105a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:56.449994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1251/2495] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:54:56.454272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:56.465657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afdffa-8002-4185-98f4-5844ba20d793 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:56.470652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:56.472420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:56.472714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:56.473021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] PATH_INFO: `/v3/projects/eb4ce2a9b2e94ca4a00b63df1fbea4e5/users/3a43b374a97f499e9cb4fa777895d038/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:56.475257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4ce2a9b2e94ca4a00b63df1fbea4e5, user_id=3a43b374a97f499e9cb4fa777895d038, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:56.477945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:56.493072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1245/2496] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:54:56 2022] PUT /identity/v3/projects/eb4ce2a9b2e94ca4a00b63df1fbea4e5/users/3a43b374a97f499e9cb4fa777895d038/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:56.507950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295a6b8-e1db-4572-a6de-34e5c3978c17 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:56.508299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295a6b8-e1db-4572-a6de-34e5c3978c17 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:56.508586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295a6b8-e1db-4572-a6de-34e5c3978c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:56.549427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7295a6b8-e1db-4572-a6de-34e5c3978c17 None None] MFA Rules not processed for user `3a43b374a97f499e9cb4fa777895d038`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:56.550690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7295a6b8-e1db-4572-a6de-34e5c3978c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:56.646361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1252/2497] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:56.657920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:56.660872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:56.737514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981bfd-6a2a-4d46-9e27-7db1c108c762 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:56.741952 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05efd1-2d75-4715-a76d-2cd5ef93fd0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:56.742625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05efd1-2d75-4715-a76d-2cd5ef93fd0a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:56.742946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05efd1-2d75-4715-a76d-2cd5ef93fd0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:56.743232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05efd1-2d75-4715-a76d-2cd5ef93fd0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:56.754242 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05efd1-2d75-4715-a76d-2cd5ef93fd0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:56.755854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05efd1-2d75-4715-a76d-2cd5ef93fd0a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:56.766702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1246/2498] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:56.848681 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295a6b8-e1db-4572-a6de-34e5c3978c17 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:56.933501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295a6b8-e1db-4572-a6de-34e5c3978c17 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:56.940523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:56.942598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:56.942598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:56.942598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] PATH_INFO: `/v3/users/56563b7e1e804dcd9cf8242d09af1b5e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:56.943666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56563b7e1e804dcd9cf8242d09af1b5e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:56.945800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:57.190528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56563b7e1e804dcd9cf8242d09af1b5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:57.206577 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56563b7e1e804dcd9cf8242d09af1b5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:57.269232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56563b7e1e804dcd9cf8242d09af1b5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:57.269952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56563b7e1e804dcd9cf8242d09af1b5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:57.270634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56563b7e1e804dcd9cf8242d09af1b5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:57.272187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:57.285876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1253/2499] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:56 2022] DELETE /identity/v3/users/56563b7e1e804dcd9cf8242d09af1b5e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:57.291275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:57.291560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:57.291906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:57.311526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:57.312765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:57.385079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1247/2500] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:54:57.396309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:57.398377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:57.466278 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b36451-6bc6-44fa-9ea5-b29af4fed597 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:57.472299 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:57.473105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:57.473420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:57.473755 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:57.485119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:57.487115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:57.505333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1254/2501] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:54:58.083799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:58.085512 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:58.109204 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca092f2-9fd2-455d-a8fc-6d1cdef1f7f4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:58.115082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:58.116004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:58.116314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:58.116597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] PATH_INFO: `/v3/projects/584fdec4218d4144915cf0be579f7353` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:58.117897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=584fdec4218d4144915cf0be579f7353)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:58.119971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:58.133373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:58.286011 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584fdec4218d4144915cf0be579f7353', 'request_id': 'req-e0e12708-c1a8-4181-a440-e94e014cde96'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.292997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584fdec4218d4144915cf0be579f7353', 'request_id': 'req-e0e12708-c1a8-4181-a440-e94e014cde96'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.400873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584fdec4218d4144915cf0be579f7353', 'request_id': 'req-e0e12708-c1a8-4181-a440-e94e014cde96'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.413976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] The token cache is being invalidate because project 584fdec4218d4144915cf0be579f7353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:58.414273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.415027 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.416686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1248/2502] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:58 2022] DELETE /identity/v3/projects/584fdec4218d4144915cf0be579f7353 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:58.420852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:58.432343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543369e-123e-42b8-879e-cbf3d2a28115 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:58.501448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:58.502666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:58.503110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:58.503525 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] PATH_INFO: `/v3/users/3a43b374a97f499e9cb4fa777895d038` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:58.504820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a43b374a97f499e9cb4fa777895d038)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:58.506926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:58.730993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a43b374a97f499e9cb4fa777895d038'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.750701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a43b374a97f499e9cb4fa777895d038'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.835666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a43b374a97f499e9cb4fa777895d038'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.836448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a43b374a97f499e9cb4fa777895d038'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.837148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dd7efbd8-7b4e-49f9-b625-a27a2766c2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a43b374a97f499e9cb4fa777895d038'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:58.851734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1255/2503] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:54:58 2022] DELETE /identity/v3/users/3a43b374a97f499e9cb4fa777895d038 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:59.311403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:59.313394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:59.329298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e12708-c1a8-4181-a440-e94e014cde96 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:59.391537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:59.392426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:59.392711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:59.393827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] PATH_INFO: `/v3/projects/eb4ce2a9b2e94ca4a00b63df1fbea4e5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:59.395101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4ce2a9b2e94ca4a00b63df1fbea4e5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:59.397212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:59.538038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4ce2a9b2e94ca4a00b63df1fbea4e5', 'request_id': 'req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:59.544356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4ce2a9b2e94ca4a00b63df1fbea4e5', 'request_id': 'req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:59.652503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4ce2a9b2e94ca4a00b63df1fbea4e5', 'request_id': 'req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:59.663916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] The token cache is being invalidate because project eb4ce2a9b2e94ca4a00b63df1fbea4e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:54:59.666744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:59.666744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b48b45ad-b8ea-45d2-8e36-cbb83fbd6f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:54:59.666744 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1249/2504] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:54:59 2022] DELETE /identity/v3/projects/eb4ce2a9b2e94ca4a00b63df1fbea4e5 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:59.671489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872eb3-ac8f-4760-8252-1fcb1d33e4ae None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:59.671960 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872eb3-ac8f-4760-8252-1fcb1d33e4ae None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:59.672077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872eb3-ac8f-4760-8252-1fcb1d33e4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:59.693004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f5872eb3-ac8f-4760-8252-1fcb1d33e4ae None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:59.694207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f5872eb3-ac8f-4760-8252-1fcb1d33e4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:59.699531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:59.753671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1256/2505] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:54:59.761672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01b69-1868-49e7-8c47-03c4836743c5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:59.761968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01b69-1868-49e7-8c47-03c4836743c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:59.762252 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01b69-1868-49e7-8c47-03c4836743c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:59.782493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-56b01b69-1868-49e7-8c47-03c4836743c5 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:54:59.783972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-56b01b69-1868-49e7-8c47-03c4836743c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:54:59.792078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1250/2506] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:54:59.796471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872eb3-ac8f-4760-8252-1fcb1d33e4ae None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:59.808615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872eb3-ac8f-4760-8252-1fcb1d33e4ae None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:59.816147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:59.816147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:59.816147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:59.816147 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:59.816617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:54:59.816959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:59.818613 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:54:59.842743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1257/2507] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:54:59.845174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b01b69-1868-49e7-8c47-03c4836743c5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:54:59.855743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b01b69-1868-49e7-8c47-03c4836743c5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:54:59.860677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:54:59.861372 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:54:59.862477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:54:59.867378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:54:59.867378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:54:59.867378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:00.082445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1251/2508] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:54:59 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:00.086967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:00.098326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d2983-2ffc-4b82-810c-16cb8d1861f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:00.156926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:00.157655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:00.157949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:00.158237 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:00.159534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:00.161580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:00.298364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1258/2509] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:00 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:00.302660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:00.308043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:00.313507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71540d9-fd4c-4930-ae5d-ad728f09368a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:00.318493 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:00.319266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:00.319554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:00.319834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:00.320576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:00.322225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:00.333750 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1252/2510] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:00.338562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:00.349607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ecad3-5a15-4b5c-ab46-c35d80435386 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:00.354213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f2d48-6657-4770-b118-7469c24ff760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:00.354865 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f2d48-6657-4770-b118-7469c24ff760 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:00.355148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f2d48-6657-4770-b118-7469c24ff760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:00.355422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f2d48-6657-4770-b118-7469c24ff760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:00.356140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f2d48-6657-4770-b118-7469c24ff760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:00.357798 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f2d48-6657-4770-b118-7469c24ff760 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:00.368160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1259/2511] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:00.372893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:00.382910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de45a-5b99-4dfb-b671-fab498981341 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:00.395597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:00.395597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:00.395597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:00.395597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] PATH_INFO: `/v3/projects/2f414be2fbd74c55a940b84f0d186ced/users/9e55ec25643f41f09b97ff43c8bea514/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:00.404884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f414be2fbd74c55a940b84f0d186ced, user_id=9e55ec25643f41f09b97ff43c8bea514, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:00.407569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:00.538937 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1253/2512] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:00 2022] PUT /identity/v3/projects/2f414be2fbd74c55a940b84f0d186ced/users/9e55ec25643f41f09b97ff43c8bea514/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:00.550970 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:00.551259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:00.551540 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:00.581658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] MFA Rules not processed for user `9e55ec25643f41f09b97ff43c8bea514`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:00.582966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:00.649648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1260/2513] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:00.661457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:00.663162 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:00.724743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da077f-26f4-486d-98f6-1325dbd71bea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:00.728451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898ecb0-92ea-4c28-b77c-bea78e8d19b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:00.729147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ecb0-92ea-4c28-b77c-bea78e8d19b3 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:00.729523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ecb0-92ea-4c28-b77c-bea78e8d19b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:00.729794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ecb0-92ea-4c28-b77c-bea78e8d19b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:00.739261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898ecb0-92ea-4c28-b77c-bea78e8d19b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:00.740701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898ecb0-92ea-4c28-b77c-bea78e8d19b3 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:00.750118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1254/2514] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:01.229703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:01.231100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:01.241420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:01.299491 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c6bd8-cf3d-41da-ab5f-0fb4789a814f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:01.304102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:01.305000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:01.305285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:01.305560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] PATH_INFO: `/v3/users/efd75526a1a44cb58329f4548c37af4f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:01.306754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd75526a1a44cb58329f4548c37af4f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:01.308690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:01.586122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd75526a1a44cb58329f4548c37af4f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:01.600824 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd75526a1a44cb58329f4548c37af4f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:01.812528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd75526a1a44cb58329f4548c37af4f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:01.816002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd75526a1a44cb58329f4548c37af4f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:01.816876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd75526a1a44cb58329f4548c37af4f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:01.834583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1261/2515] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:01 2022] DELETE /identity/v3/users/efd75526a1a44cb58329f4548c37af4f => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:01.839742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:01.840236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:01.840236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:01.865418 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:01.865950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:01.958527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1255/2516] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:01.959217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:01.959217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:02.026479 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73ca7-1d60-4cb3-abab-bb74a5fe4c55 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:02.039179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc580a-db88-4e19-b4e2-a8c78ed2deed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:02.039791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc580a-db88-4e19-b4e2-a8c78ed2deed service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:02.040078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc580a-db88-4e19-b4e2-a8c78ed2deed service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:02.040352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc580a-db88-4e19-b4e2-a8c78ed2deed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:02.053717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc580a-db88-4e19-b4e2-a8c78ed2deed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:02.055268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc580a-db88-4e19-b4e2-a8c78ed2deed service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:02.068254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1262/2517] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:02.694881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:02.696290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:02.710159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd0dac-ac14-44d3-858e-8f0935f9c4b4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:02.717419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:02.718114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:02.718400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:02.718677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] PATH_INFO: `/v3/projects/0b2a8b6cbcf84cdf8ecdba5de18eff7f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:02.720029 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2a8b6cbcf84cdf8ecdba5de18eff7f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:02.723093 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:02.882922 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2a8b6cbcf84cdf8ecdba5de18eff7f', 'request_id': 'req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:02.889669 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2a8b6cbcf84cdf8ecdba5de18eff7f', 'request_id': 'req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:03.063636 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2a8b6cbcf84cdf8ecdba5de18eff7f', 'request_id': 'req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:03.071369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:03.079123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] The token cache is being invalidate because project 0b2a8b6cbcf84cdf8ecdba5de18eff7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:03.079573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:03.080338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1985fb31-8ed1-4b87-bf58-cde16a5d22bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:03.082157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1256/2518] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:02 2022] DELETE /identity/v3/projects/0b2a8b6cbcf84cdf8ecdba5de18eff7f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:03.087076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbff75-bb73-422d-886c-174233449b51 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.092523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbff75-bb73-422d-886c-174233449b51 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.092523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbff75-bb73-422d-886c-174233449b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.107179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-13cbff75-bb73-422d-886c-174233449b51 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:03.108430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13cbff75-bb73-422d-886c-174233449b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:03.137067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:03.172609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1263/2519] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:03.181318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99effd-0292-4acf-bad4-e2e52912ed01 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.181610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99effd-0292-4acf-bad4-e2e52912ed01 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.181897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99effd-0292-4acf-bad4-e2e52912ed01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.202427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e99effd-0292-4acf-bad4-e2e52912ed01 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:03.204102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e99effd-0292-4acf-bad4-e2e52912ed01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:03.213057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1257/2520] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:03.217615 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbff75-bb73-422d-886c-174233449b51 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:03.228148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbff75-bb73-422d-886c-174233449b51 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:03.233432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:03.233989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.234413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.234696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.235466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:03.235776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:03.238179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:03.252155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1264/2521] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:55:03.257501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99effd-0292-4acf-bad4-e2e52912ed01 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:03.266638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99effd-0292-4acf-bad4-e2e52912ed01 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:03.272219 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:03.272928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.273215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.273494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.274217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:03.276156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:03.391622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1258/2522] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:03 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:03.396708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:03.408714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04ab7f-4764-4caa-bd42-0f760a25e513 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:03.472348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:03.472557 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:03.473574 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.474000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.474113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:03.474398 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.475190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:03.478154 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:03.574149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5ace-7205-4c5f-af5f-26945a36f6cd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:03.579650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:03.580385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.580852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.581137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.583243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:03.589593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1265/2523] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:03 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:03.591039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:03.596596 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:03.598888 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:03.606777 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7628c5-294b-4d5a-ab89-200970d5628c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:03.611053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1259/2524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:03.612965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:03.613788 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.614072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.614346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.615090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:03.616712 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:03.628131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1266/2525] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:03.632541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:03.647221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbccaba-983d-4a37-89ec-1949506df4c6 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:03.652956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b14fa3-ff47-43f4-98a4-038eca6357c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:03.653771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b14fa3-ff47-43f4-98a4-038eca6357c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.654060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b14fa3-ff47-43f4-98a4-038eca6357c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.654332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b14fa3-ff47-43f4-98a4-038eca6357c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.655064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b14fa3-ff47-43f4-98a4-038eca6357c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:03.656678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b14fa3-ff47-43f4-98a4-038eca6357c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:03.672843 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1260/2526] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:03.674748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:03.687337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756d872-c45a-4a2f-aa41-f64b960f8edf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:03.692381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:03.693321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.693609 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.693887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] PATH_INFO: `/v3/projects/39b0658858924479a3be4aa549cd6ed4/users/dc871f648e864e1684b9680d6eed5805/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.712826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b0658858924479a3be4aa549cd6ed4, user_id=dc871f648e864e1684b9680d6eed5805, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:03.714908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:03.885228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1267/2527] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:03 2022] PUT /identity/v3/projects/39b0658858924479a3be4aa549cd6ed4/users/dc871f648e864e1684b9680d6eed5805/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:03.897332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d0055-7fa9-44e6-b313-feb8e515a622 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:03.897871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d0055-7fa9-44e6-b313-feb8e515a622 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:03.897871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d0055-7fa9-44e6-b313-feb8e515a622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:03.926567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bb0d0055-7fa9-44e6-b313-feb8e515a622 None None] MFA Rules not processed for user `dc871f648e864e1684b9680d6eed5805`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:03.927834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bb0d0055-7fa9-44e6-b313-feb8e515a622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:04.022656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1261/2528] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:04.027413 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:04.103668 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f0c7b-ed37-405c-93b4-f0499356c9dc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:04.109242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:04.109919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:04.110264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:04.110594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:04.111344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:04.113885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:04.131248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:04.263986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1268/2529] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:04 2022] POST /identity/v3/projects => generated 379 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:04.265323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d0055-7fa9-44e6-b313-feb8e515a622 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:04.277506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d0055-7fa9-44e6-b313-feb8e515a622 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:04.288866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:04.362010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:04.362699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:04.362981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:04.363257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:04.364003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:04.366158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:04.493387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:04.495299 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1262/2530] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:04 2022] POST /identity/v3/users => generated 296 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:04.505401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389966-50c5-4a18-8193-b71074fe74df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:04.514562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:04.515250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:04.515536 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:04.515811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:04.516559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:04.519142 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:04.531447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1269/2531] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:04.536358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:04.546583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684c67d-b448-4949-9525-65ea1b7313df admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:04.551753 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:04.552660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:04.553009 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:04.553278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] PATH_INFO: `/v3/projects/dbf67898da8d446eb901178aced22cce/users/84547c39614443aeb0eca257e90511f7/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:04.582547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf67898da8d446eb901178aced22cce, user_id=84547c39614443aeb0eca257e90511f7, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:04.585691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:04.742856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1263/2532] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:04 2022] PUT /identity/v3/projects/dbf67898da8d446eb901178aced22cce/users/84547c39614443aeb0eca257e90511f7/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:04.746740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:04.767239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10000d5-63fc-47e3-a0a2-94ca77d6c0b4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:04.863847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f595e8-87f1-49a2-8188-1bfe4b28a23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:04.864705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f595e8-87f1-49a2-8188-1bfe4b28a23f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:04.865159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f595e8-87f1-49a2-8188-1bfe4b28a23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:04.865455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f595e8-87f1-49a2-8188-1bfe4b28a23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:04.866231 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f595e8-87f1-49a2-8188-1bfe4b28a23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:04.867984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f595e8-87f1-49a2-8188-1bfe4b28a23f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:04.884720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1270/2533] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:04.889084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:04.900168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e370d36-b06b-4759-8260-4068051d2f0a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:04.905099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:04.906003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:04.906279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:04.906615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] PATH_INFO: `/v3/projects/dbf67898da8d446eb901178aced22cce/users/84547c39614443aeb0eca257e90511f7/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:04.908615 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf67898da8d446eb901178aced22cce, user_id=84547c39614443aeb0eca257e90511f7, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:04.911944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:04.929846 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1264/2534] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:04 2022] PUT /identity/v3/projects/dbf67898da8d446eb901178aced22cce/users/84547c39614443aeb0eca257e90511f7/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:04.943685 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed032e0-c6c1-403f-9e66-cbb18841ba89 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:04.944382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed032e0-c6c1-403f-9e66-cbb18841ba89 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:04.944750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed032e0-c6c1-403f-9e66-cbb18841ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:04.980135 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0ed032e0-c6c1-403f-9e66-cbb18841ba89 None None] MFA Rules not processed for user `84547c39614443aeb0eca257e90511f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:04.981446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ed032e0-c6c1-403f-9e66-cbb18841ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:05.090339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1271/2535] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:05.109891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:05.114579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:05.184047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafc886-d66a-4308-a3c4-bf2f589b7221 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:05.192360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558496dc-956b-4802-9333-479989458d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:05.193271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558496dc-956b-4802-9333-479989458d17 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:05.200099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558496dc-956b-4802-9333-479989458d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:05.200099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558496dc-956b-4802-9333-479989458d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:05.204435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558496dc-956b-4802-9333-479989458d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:05.206141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558496dc-956b-4802-9333-479989458d17 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:05.217224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1265/2536] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:05.655273 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed032e0-c6c1-403f-9e66-cbb18841ba89 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:05.666337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed032e0-c6c1-403f-9e66-cbb18841ba89 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:05.670314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:05.671039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:05.671379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:05.671680 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:05.744594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:05.746264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:05.760292 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1272/2537] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:10.522154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558496dc-956b-4802-9333-479989458d17 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:10.524022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-558496dc-956b-4802-9333-479989458d17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:10.587145 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558496dc-956b-4802-9333-479989458d17 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:10.591160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:10.591813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:10.592102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:10.592382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:10.596432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:10.626120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:10.667842 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:10.669368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:10.681213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1266/2538] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:14.661743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:14.663545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:14.675599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:14.679911 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baabe20-f51a-4a66-bf76-bc18542a78a9 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:14.690533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:14.699564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:14.699564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:14.699564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] PATH_INFO: `/v3/users/7e032c8e5b024fa7a41cd1090e0f1ada` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:14.699564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e032c8e5b024fa7a41cd1090e0f1ada)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:14.699564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:15.000177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e032c8e5b024fa7a41cd1090e0f1ada'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.013307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e032c8e5b024fa7a41cd1090e0f1ada'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.120170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e032c8e5b024fa7a41cd1090e0f1ada'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.120932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e032c8e5b024fa7a41cd1090e0f1ada'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.121580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-55a3b444-82a9-4e79-a7cb-ca15baed9e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e032c8e5b024fa7a41cd1090e0f1ada'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.138876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1273/2539] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:14 2022] DELETE /identity/v3/users/7e032c8e5b024fa7a41cd1090e0f1ada => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:15.465544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:15.475957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada017d9-5cde-4ff4-a998-2abb21b816df service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:15.536249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:15.537148 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:15.537432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:15.537715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] PATH_INFO: `/v3/projects/48497435674b41ccb34f6ff43026a232` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:15.538993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=48497435674b41ccb34f6ff43026a232)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:15.540930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:15.689197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48497435674b41ccb34f6ff43026a232', 'request_id': 'req-a1c7e058-193d-4721-969f-d50e755ec5ff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.696302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48497435674b41ccb34f6ff43026a232', 'request_id': 'req-a1c7e058-193d-4721-969f-d50e755ec5ff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.788253 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48497435674b41ccb34f6ff43026a232', 'request_id': 'req-a1c7e058-193d-4721-969f-d50e755ec5ff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.812266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] The token cache is being invalidate because project 48497435674b41ccb34f6ff43026a232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:15.812565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.813341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1c7e058-193d-4721-969f-d50e755ec5ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:15.815033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1267/2540] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:15 2022] DELETE /identity/v3/projects/48497435674b41ccb34f6ff43026a232 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:15.820401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b63ee-a85b-4c0f-bae5-a2d7e215dc2f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:15.820920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b63ee-a85b-4c0f-bae5-a2d7e215dc2f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:15.821128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b63ee-a85b-4c0f-bae5-a2d7e215dc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:15.839519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-607b63ee-a85b-4c0f-bae5-a2d7e215dc2f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:15.840720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-607b63ee-a85b-4c0f-bae5-a2d7e215dc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:15.901935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1274/2541] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:15.910841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296a152-f2d5-4c90-ad7c-76a37e9f6f87 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:15.920526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296a152-f2d5-4c90-ad7c-76a37e9f6f87 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:15.920526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296a152-f2d5-4c90-ad7c-76a37e9f6f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:15.931303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5296a152-f2d5-4c90-ad7c-76a37e9f6f87 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:15.932533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5296a152-f2d5-4c90-ad7c-76a37e9f6f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:15.943468 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1268/2542] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:15.947843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b63ee-a85b-4c0f-bae5-a2d7e215dc2f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:15.958762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b63ee-a85b-4c0f-bae5-a2d7e215dc2f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:15.964164 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:15.964969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:15.965258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:15.965539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:15.966298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:15.966608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:15.968246 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:15.982353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1275/2543] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:55:15.986866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296a152-f2d5-4c90-ad7c-76a37e9f6f87 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:15.996653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296a152-f2d5-4c90-ad7c-76a37e9f6f87 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:16.001331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:16.002129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:16.002477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:16.002754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:16.003466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:16.005440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:16.379916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1269/2544] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:15 2022] POST /identity/v3/projects => generated 383 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:16.384283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:16.396047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2ed49-bdae-4ad5-afad-b9671b0b386c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:16.461038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:16.461866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:16.462221 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:16.462687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:16.463726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:16.465777 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:16.475592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:16.641461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1276/2545] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:16 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:16.645641 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:16.659655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16318852-6726-49dd-a993-afef81d11047 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:16.664541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:16.665284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:16.665571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:16.665847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:16.666579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:16.668180 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:16.679896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1270/2546] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:16.684460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:16.695477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcf348-e022-45f9-9576-cd0e6a364d37 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:16.700198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30f43a-b3ca-4cf5-a7a1-a1d7627bf559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:16.700897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30f43a-b3ca-4cf5-a7a1-a1d7627bf559 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:16.706053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30f43a-b3ca-4cf5-a7a1-a1d7627bf559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:16.706053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30f43a-b3ca-4cf5-a7a1-a1d7627bf559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:16.706053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30f43a-b3ca-4cf5-a7a1-a1d7627bf559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:16.706053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30f43a-b3ca-4cf5-a7a1-a1d7627bf559 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:16.715905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1277/2547] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:16.720198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:16.731762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20dd1b-2e61-4a23-8d46-1874eb594a62 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:16.736752 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:16.737674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:16.737972 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:16.738247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] PATH_INFO: `/v3/projects/61a2df2a0e8e48f1bdc613750e16eca6/users/2a536e4b20f04a8bbdcd77c6d8bfcd16/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:16.741552 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:16.753425 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a2df2a0e8e48f1bdc613750e16eca6, user_id=2a536e4b20f04a8bbdcd77c6d8bfcd16, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:16.756101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:16.841716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1271/2548] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:16 2022] PUT /identity/v3/projects/61a2df2a0e8e48f1bdc613750e16eca6/users/2a536e4b20f04a8bbdcd77c6d8bfcd16/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:16.852758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89214c40-2428-4405-a84f-f521e15725bb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:16.853304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89214c40-2428-4405-a84f-f521e15725bb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:16.853411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89214c40-2428-4405-a84f-f521e15725bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:16.887068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-89214c40-2428-4405-a84f-f521e15725bb None None] MFA Rules not processed for user `2a536e4b20f04a8bbdcd77c6d8bfcd16`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:16.888258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-89214c40-2428-4405-a84f-f521e15725bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:16.964276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1278/2549] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:16.976292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:16.978123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:17.092044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c1c0e-7c23-49c9-b1b4-60a36ac872d6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:17.099440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:17.100118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:17.100411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:17.100695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:17.109793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:17.112406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:17.122876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1272/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:17.553508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89214c40-2428-4405-a84f-f521e15725bb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:17.554952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-89214c40-2428-4405-a84f-f521e15725bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:17.616176 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89214c40-2428-4405-a84f-f521e15725bb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:17.619776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9e16b-1849-4e69-8f38-76e1f62fc8bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:17.620422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9e16b-1849-4e69-8f38-76e1f62fc8bc service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:17.620713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9e16b-1849-4e69-8f38-76e1f62fc8bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:17.621016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9e16b-1849-4e69-8f38-76e1f62fc8bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:17.622852 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ab9e16b-1849-4e69-8f38-76e1f62fc8bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:17.688648 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9e16b-1849-4e69-8f38-76e1f62fc8bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:17.690173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9e16b-1849-4e69-8f38-76e1f62fc8bc service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:17.699296 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1279/2551] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:19.749735 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:19.751336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:19.831207 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce2e08-5ec8-4461-b8b7-fb2e56cca23c service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:19.836267 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:19.837387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:19.837674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:19.838212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] PATH_INFO: `/v3/users/dc871f648e864e1684b9680d6eed5805` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:19.839630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc871f648e864e1684b9680d6eed5805)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:19.841691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:20.171119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:20.173424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc871f648e864e1684b9680d6eed5805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:20.185989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc871f648e864e1684b9680d6eed5805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:20.200153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc871f648e864e1684b9680d6eed5805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:20.285875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc871f648e864e1684b9680d6eed5805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:20.286532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc871f648e864e1684b9680d6eed5805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:20.293291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1273/2552] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:19 2022] DELETE /identity/v3/users/dc871f648e864e1684b9680d6eed5805 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:20.298046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:20.299234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:20.299234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:20.316923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:20.318599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:20.339875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:20.378655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1280/2553] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:20.388620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:20.390108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:20.449396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27493dc5-da06-41c2-90a7-945228d2ad5c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:20.453076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:20.453662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:20.453946 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:20.454217 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:20.465683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:20.467138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:20.477722 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1274/2554] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:20.945081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:20.946587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:20.959356 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccdd47-a7c4-4397-a3ac-711836cc519c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:20.964444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:20.965417 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:20.965710 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:20.965990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] PATH_INFO: `/v3/projects/39b0658858924479a3be4aa549cd6ed4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:20.967224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b0658858924479a3be4aa549cd6ed4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:20.969230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:21.087234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b0658858924479a3be4aa549cd6ed4', 'request_id': 'req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.177257 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b0658858924479a3be4aa549cd6ed4', 'request_id': 'req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.184446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b0658858924479a3be4aa549cd6ed4', 'request_id': 'req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.197222 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] The token cache is being invalidate because project 39b0658858924479a3be4aa549cd6ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:21.197529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.198348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.200363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1281/2555] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:20 2022] DELETE /identity/v3/projects/39b0658858924479a3be4aa549cd6ed4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:21.204334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:21.216282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1792-3038-498d-a18d-eb538c0fb090 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:21.275554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:21.276427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:21.276715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:21.277089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] PATH_INFO: `/v3/users/84547c39614443aeb0eca257e90511f7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:21.278282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84547c39614443aeb0eca257e90511f7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:21.280226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:21.528837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84547c39614443aeb0eca257e90511f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.536921 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84547c39614443aeb0eca257e90511f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.545233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:21.549889 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84547c39614443aeb0eca257e90511f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.640325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84547c39614443aeb0eca257e90511f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.641013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0a7ba86-4923-4951-8097-94fea38a29f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84547c39614443aeb0eca257e90511f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:21.647654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1275/2556] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:21 2022] DELETE /identity/v3/users/84547c39614443aeb0eca257e90511f7 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:22.150702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:22.152298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:22.166340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4e84d-0de2-4baf-a65c-4a2ac683cf29 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:22.228814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:22.230014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:22.230295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:22.230637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] PATH_INFO: `/v3/projects/dbf67898da8d446eb901178aced22cce` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:22.232746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf67898da8d446eb901178aced22cce)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:22.234797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:22.373205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf67898da8d446eb901178aced22cce', 'request_id': 'req-875508d0-b2b4-40ed-99e8-6c3c48798589'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:22.376159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:22.463473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf67898da8d446eb901178aced22cce', 'request_id': 'req-875508d0-b2b4-40ed-99e8-6c3c48798589'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:22.470528 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf67898da8d446eb901178aced22cce', 'request_id': 'req-875508d0-b2b4-40ed-99e8-6c3c48798589'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:22.480850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] The token cache is being invalidate because project dbf67898da8d446eb901178aced22cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:22.481144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:22.481874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-875508d0-b2b4-40ed-99e8-6c3c48798589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:22.483690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1282/2557] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:22 2022] DELETE /identity/v3/projects/dbf67898da8d446eb901178aced22cce => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:22.488878 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8c624-7998-4bf1-8c1f-95c80d00dbab None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:22.489470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8c624-7998-4bf1-8c1f-95c80d00dbab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:22.489470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8c624-7998-4bf1-8c1f-95c80d00dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:22.507995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9da8c624-7998-4bf1-8c1f-95c80d00dbab None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:22.509256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9da8c624-7998-4bf1-8c1f-95c80d00dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:22.575131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1276/2558] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:22.585071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c2805-dd32-48f7-b73e-45aa64b09015 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:22.585379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c2805-dd32-48f7-b73e-45aa64b09015 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:22.585640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c2805-dd32-48f7-b73e-45aa64b09015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:22.604920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b06c2805-dd32-48f7-b73e-45aa64b09015 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:22.606193 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b06c2805-dd32-48f7-b73e-45aa64b09015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:22.614341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1283/2559] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:22.622436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8c624-7998-4bf1-8c1f-95c80d00dbab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:22.632804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8c624-7998-4bf1-8c1f-95c80d00dbab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:22.637413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:22.638077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:22.638361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:22.638648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:22.639395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:22.639703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:22.641360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:22.658276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1277/2560] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:55:22.660807 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c2805-dd32-48f7-b73e-45aa64b09015 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:22.671191 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c2805-dd32-48f7-b73e-45aa64b09015 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:22.677811 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:22.678588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:22.678961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:22.679428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:22.680080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:22.682229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:22.832379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1284/2561] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:22 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:22.836606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:22.847728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90b65e-7d48-412c-96d3-627a5c457803 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:22.910003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:22.910698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:22.911133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:22.911255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:22.912133 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:22.914125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.041173 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1278/2562] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:22 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:23.045497 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.055703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a204f2-ceef-4f9d-bce5-2ce4767311e6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.060408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.061049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.061333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.061608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.062338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.064617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.075559 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1285/2563] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:23.080359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.090845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42261ea-be22-4903-98fc-ba6d04818aae admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.095865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c612e74-4f6b-49eb-8624-64d5a3f8e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.102505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c612e74-4f6b-49eb-8624-64d5a3f8e717 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.102505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c612e74-4f6b-49eb-8624-64d5a3f8e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.102505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c612e74-4f6b-49eb-8624-64d5a3f8e717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.102505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c612e74-4f6b-49eb-8624-64d5a3f8e717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.102505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c612e74-4f6b-49eb-8624-64d5a3f8e717 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.110947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1279/2564] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:23.115968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.125872 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29da12-2e18-42d7-82bf-7acd7048f967 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.131480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.136907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.136907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.136907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] PATH_INFO: `/v3/projects/abbff9e15bad46c4a13054f9a8e7c758/users/a346c9634e6144e982812662451ba6e5/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.148382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbff9e15bad46c4a13054f9a8e7c758, user_id=a346c9634e6144e982812662451ba6e5, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.151298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.275420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1286/2565] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:23 2022] PUT /identity/v3/projects/abbff9e15bad46c4a13054f9a8e7c758/users/a346c9634e6144e982812662451ba6e5/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:23.286422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d20c7-4f70-4095-ba5a-5708c2b1ccfa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.286713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d20c7-4f70-4095-ba5a-5708c2b1ccfa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.287062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d20c7-4f70-4095-ba5a-5708c2b1ccfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.319216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-159d20c7-4f70-4095-ba5a-5708c2b1ccfa None None] MFA Rules not processed for user `a346c9634e6144e982812662451ba6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:23.319216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-159d20c7-4f70-4095-ba5a-5708c2b1ccfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:23.365964 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:23.394318 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1280/2566] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:23.399018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.461638 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a7921-7974-4a48-8258-41b5899bc303 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.466653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.467317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.467604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.467879 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.468578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.470748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.562309 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1287/2567] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:23 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:23.567272 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d20c7-4f70-4095-ba5a-5708c2b1ccfa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.577988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d20c7-4f70-4095-ba5a-5708c2b1ccfa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.636403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.637129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.637600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.637890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.638592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.640715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.726816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1281/2568] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:23 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:23.730843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.742979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd6cb6-c4dd-45f6-ac8a-1b368e6e0705 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.748194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.748955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.749242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.749527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.750310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.752589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.763622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1288/2569] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:23.767939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.778428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a9af-d4d5-497e-85c0-f8026d0bc977 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.783399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.784282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.784573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.784891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] PATH_INFO: `/v3/projects/7b8b62003c0a4a45947d0537cd7d386c/users/73776a6bd97743bfb392ba4fbfc6e224/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.801076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8b62003c0a4a45947d0537cd7d386c, user_id=73776a6bd97743bfb392ba4fbfc6e224, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.803778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.910041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1282/2570] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:23 2022] PUT /identity/v3/projects/7b8b62003c0a4a45947d0537cd7d386c/users/73776a6bd97743bfb392ba4fbfc6e224/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:23.914581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:23.925545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd802d17-34c4-4096-b581-c5984cbcd05c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:23.974834 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:23.985129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05d095-1029-4b3e-9662-a61aea1f1959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:23.986339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05d095-1029-4b3e-9662-a61aea1f1959 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:23.986633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05d095-1029-4b3e-9662-a61aea1f1959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:23.986976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05d095-1029-4b3e-9662-a61aea1f1959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:23.987695 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05d095-1029-4b3e-9662-a61aea1f1959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:23.989339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05d095-1029-4b3e-9662-a61aea1f1959 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:23.999122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1289/2571] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:24.003558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:24.013903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f61b3-a90d-4134-b78c-804eb3a7d983 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:24.018994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:24.019874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:24.020157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:24.020430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] PATH_INFO: `/v3/projects/7b8b62003c0a4a45947d0537cd7d386c/users/73776a6bd97743bfb392ba4fbfc6e224/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:24.022573 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8b62003c0a4a45947d0537cd7d386c, user_id=73776a6bd97743bfb392ba4fbfc6e224, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:24.025440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:24.036630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1283/2572] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:24 2022] PUT /identity/v3/projects/7b8b62003c0a4a45947d0537cd7d386c/users/73776a6bd97743bfb392ba4fbfc6e224/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:24.047886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:24.048820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:24.049130 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:24.077555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] MFA Rules not processed for user `73776a6bd97743bfb392ba4fbfc6e224`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:24.078933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:24.151146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1290/2573] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:24.163612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:24.165398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:24.224660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f900fb0-3f1f-4e5e-b177-3e3832ac86e1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:24.228869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee6652-15b3-4f0a-a9bb-ade5bd1c201b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:24.229542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6652-15b3-4f0a-a9bb-ade5bd1c201b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:24.229837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6652-15b3-4f0a-a9bb-ade5bd1c201b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:24.230115 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6652-15b3-4f0a-a9bb-ade5bd1c201b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:24.278306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:24.290718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee6652-15b3-4f0a-a9bb-ade5bd1c201b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:24.292260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee6652-15b3-4f0a-a9bb-ade5bd1c201b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:24.302268 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1284/2574] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:27.557876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:27.559466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:27.634702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376b8b3-c879-4060-bdc5-d8f4b3d266b1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:27.639663 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:27.640576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:27.640898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:27.641179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] PATH_INFO: `/v3/users/9e55ec25643f41f09b97ff43c8bea514` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:27.642987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e55ec25643f41f09b97ff43c8bea514)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:27.644999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:27.911892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e55ec25643f41f09b97ff43c8bea514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:27.924899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e55ec25643f41f09b97ff43c8bea514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:28.063745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e55ec25643f41f09b97ff43c8bea514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:28.065119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e55ec25643f41f09b97ff43c8bea514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:28.065737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e55ec25643f41f09b97ff43c8bea514'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:28.068744 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:28.080959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1291/2575] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:27 2022] DELETE /identity/v3/users/9e55ec25643f41f09b97ff43c8bea514 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:28.085618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb859605-9901-427e-8109-60f5a5dfd8e8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:28.086087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb859605-9901-427e-8109-60f5a5dfd8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:28.086333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb859605-9901-427e-8109-60f5a5dfd8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:28.105576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cb859605-9901-427e-8109-60f5a5dfd8e8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:28.106803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb859605-9901-427e-8109-60f5a5dfd8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:28.168986 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1285/2576] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:28.179518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:28.181003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:28.240620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759b740-f8b0-4aeb-a5d3-f372e1f0f7bc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:28.244285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:28.244986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:28.245481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:28.245919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:28.257437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:28.259026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:28.269133 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1292/2577] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:28.750537 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dc730-aa53-486b-935d-c433a0f6762b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:28.751030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dc730-aa53-486b-935d-c433a0f6762b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:28.751509 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dc730-aa53-486b-935d-c433a0f6762b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:28.771669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:28.774612 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d00dc730-aa53-486b-935d-c433a0f6762b None None] MFA Rules not processed for user `eb483b1aee944da8910dd7468a0ef4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:28.774764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:28.775874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d00dc730-aa53-486b-935d-c433a0f6762b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:28.790760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594b41-c2e7-489a-9c2c-370a813a8044 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:28.804874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:28.805998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:28.812990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:28.812990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] PATH_INFO: `/v3/projects/2f414be2fbd74c55a940b84f0d186ced` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:28.812990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f414be2fbd74c55a940b84f0d186ced)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:28.812990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:28.848737 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1286/2578] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:28.974942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f414be2fbd74c55a940b84f0d186ced', 'request_id': 'req-86b57d86-8c29-4d4b-804c-b08334fc2783'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.063698 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f414be2fbd74c55a940b84f0d186ced', 'request_id': 'req-86b57d86-8c29-4d4b-804c-b08334fc2783'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.074283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f414be2fbd74c55a940b84f0d186ced', 'request_id': 'req-86b57d86-8c29-4d4b-804c-b08334fc2783'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.085761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] The token cache is being invalidate because project 2f414be2fbd74c55a940b84f0d186ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:29.086052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.086800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86b57d86-8c29-4d4b-804c-b08334fc2783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.088703 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1293/2579] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:28 2022] DELETE /identity/v3/projects/2f414be2fbd74c55a940b84f0d186ced => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:29.094450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d001f-78cf-4898-9948-0a86a3def04f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.095415 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d001f-78cf-4898-9948-0a86a3def04f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.095569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d001f-78cf-4898-9948-0a86a3def04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.117574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-711d001f-78cf-4898-9948-0a86a3def04f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:29.119004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-711d001f-78cf-4898-9948-0a86a3def04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:29.216942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1287/2580] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:29.223985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.224281 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.224565 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.252271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:29.253727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:29.264150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1294/2581] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:29.268953 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d001f-78cf-4898-9948-0a86a3def04f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.277699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.279382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:29.281770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d001f-78cf-4898-9948-0a86a3def04f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.288063 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.290855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.290855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.290855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.290855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:29.290855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.292814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.293878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22451e5b-a77b-4c87-8344-f10279f36b13 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.304495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.305427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.305714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.305989 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] PATH_INFO: `/v3/users/2a536e4b20f04a8bbdcd77c6d8bfcd16` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.307295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a536e4b20f04a8bbdcd77c6d8bfcd16)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.307471 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1288/2582] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:55:29.309640 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.312919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.324290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764caa8-acd0-461c-a409-f524c34a9841 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.331104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.331782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.332084 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.332360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.333090 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.335128 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.498335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:29.507227 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1289/2583] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:29 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:29.514627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.583089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097ac8-ba42-4904-b394-52f6379fc744 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.588398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.589211 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.589501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.589802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.590505 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.592421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.614883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a536e4b20f04a8bbdcd77c6d8bfcd16'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.628230 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a536e4b20f04a8bbdcd77c6d8bfcd16'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.630363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:29.726106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a536e4b20f04a8bbdcd77c6d8bfcd16'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.727162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a536e4b20f04a8bbdcd77c6d8bfcd16'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.728233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd3a2fdf-8dc8-4c4e-9250-5dfe84883f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a536e4b20f04a8bbdcd77c6d8bfcd16'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:29.736280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1290/2584] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:29 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:29.737716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.739403 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:29.742937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1295/2585] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:29 2022] DELETE /identity/v3/users/2a536e4b20f04a8bbdcd77c6d8bfcd16 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:29.747365 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.747660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.747971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.752674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ca4c-f0b9-485a-8905-95d49b3cd3a8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.767252 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:29.768464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:29.810408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.811199 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.811546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.811886 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.812691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.815436 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.828660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1291/2586] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:29.830457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.831583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1296/2587] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:29.849471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.849633 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f31e9-2967-4dae-9851-be28f89b0d10 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.849764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:29.849863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.849863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.849863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.849863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.851028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.852677 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.864833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1292/2588] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:29.869077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:29.879578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816c47-69c3-455a-816a-f5f2667ecfa5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.884483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.885442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.885729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.897085 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] PATH_INFO: `/v3/projects/bfeea8b2c4f04089ad8caac0c0efd57d/users/32dcff05551e4da7acf3753b5cebf041/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.901357 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfeea8b2c4f04089ad8caac0c0efd57d, user_id=32dcff05551e4da7acf3753b5cebf041, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.904146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.906867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352377b1-c52f-4654-8a9e-224a67091305 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:29.912998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e860930-8fb2-4ce9-b86c-ef2655798e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:29.913655 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e860930-8fb2-4ce9-b86c-ef2655798e66 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:29.913940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e860930-8fb2-4ce9-b86c-ef2655798e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:29.914351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e860930-8fb2-4ce9-b86c-ef2655798e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:29.925394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e860930-8fb2-4ce9-b86c-ef2655798e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:29.926848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e860930-8fb2-4ce9-b86c-ef2655798e66 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:29.936821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1297/2589] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:30.030228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1293/2590] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:29 2022] PUT /identity/v3/projects/bfeea8b2c4f04089ad8caac0c0efd57d/users/32dcff05551e4da7acf3753b5cebf041/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:30.041366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dd0c5-809d-49cc-8780-f114f9472286 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.041727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dd0c5-809d-49cc-8780-f114f9472286 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.042087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dd0c5-809d-49cc-8780-f114f9472286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.071306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-367dd0c5-809d-49cc-8780-f114f9472286 None None] MFA Rules not processed for user `32dcff05551e4da7acf3753b5cebf041`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:30.072566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-367dd0c5-809d-49cc-8780-f114f9472286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:30.139561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1298/2591] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:30.145016 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:30.199748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:30.218875 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b3cb9-f7a9-4179-88e4-bcdc408bca70 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:30.225501 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:30.226168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.226457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.226746 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.227490 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:30.229549 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:30.400847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1294/2592] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:30 2022] POST /identity/v3/projects => generated 375 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:30.402346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dd0c5-809d-49cc-8780-f114f9472286 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:30.414783 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dd0c5-809d-49cc-8780-f114f9472286 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:30.427914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:30.473359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:30.474038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.474325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.474603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.475308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:30.477288 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:30.516010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:30.518630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:30.534218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76069f56-b982-4562-a3e9-987773f215af admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:30.539433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:30.540657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.540999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.541442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] PATH_INFO: `/v3/projects/61a2df2a0e8e48f1bdc613750e16eca6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.542706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a2df2a0e8e48f1bdc613750e16eca6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:30.544704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:30.600562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1299/2593] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:30 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:30.604458 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:30.614342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523676f-dc45-441e-826e-2026f7d7d016 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:30.619813 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:30.620472 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.620758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.621068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.621844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:30.623464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:30.634803 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1300/2594] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:30.639054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:30.649527 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50823566-a608-4727-8d3d-b366553b07ab admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:30.654295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:30.655153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.655440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.655731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] PATH_INFO: `/v3/projects/77b95a8033bc4f6e9da3d5abcec71af5/users/d7e95456a1814f1d893160cb9c9d0976/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.671028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b95a8033bc4f6e9da3d5abcec71af5, user_id=d7e95456a1814f1d893160cb9c9d0976, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:30.673814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:30.818762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1301/2595] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:30 2022] PUT /identity/v3/projects/77b95a8033bc4f6e9da3d5abcec71af5/users/d7e95456a1814f1d893160cb9c9d0976/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:30.822850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:30.828196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a2df2a0e8e48f1bdc613750e16eca6', 'request_id': 'req-41154abb-b8cd-4d38-892f-f7b16aaac821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:30.835938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a2df2a0e8e48f1bdc613750e16eca6', 'request_id': 'req-41154abb-b8cd-4d38-892f-f7b16aaac821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:30.886445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597a31-f152-4a4a-a324-3ae548443d0a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:30.947199 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:30.948145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.948468 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.948690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.948827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a2df2a0e8e48f1bdc613750e16eca6', 'request_id': 'req-41154abb-b8cd-4d38-892f-f7b16aaac821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:30.949503 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:30.951363 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:30.962095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] The token cache is being invalidate because project 61a2df2a0e8e48f1bdc613750e16eca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:30.962441 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:30.963311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-41154abb-b8cd-4d38-892f-f7b16aaac821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:30.964121 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1302/2596] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:30.967818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:30.968132 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1295/2597] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:30 2022] DELETE /identity/v3/projects/61a2df2a0e8e48f1bdc613750e16eca6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:30.972809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1dc7d-a995-44d4-ae0f-e4fb8f91dae7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.973230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1dc7d-a995-44d4-ae0f-e4fb8f91dae7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.973386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:30.973614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1dc7d-a995-44d4-ae0f-e4fb8f91dae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.988569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63f14e-35d2-45cc-9bc8-5898ded41977 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:30.993383 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-81b1dc7d-a995-44d4-ae0f-e4fb8f91dae7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:30.993567 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce119657-83e7-446e-bb45-80cab2005a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:30.999504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119657-83e7-446e-bb45-80cab2005a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:30.999504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119657-83e7-446e-bb45-80cab2005a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:30.999726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-81b1dc7d-a995-44d4-ae0f-e4fb8f91dae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:30.999822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119657-83e7-446e-bb45-80cab2005a07 admin admin] PATH_INFO: `/v3/projects/77b95a8033bc4f6e9da3d5abcec71af5/users/d7e95456a1814f1d893160cb9c9d0976/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:30.999822 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce119657-83e7-446e-bb45-80cab2005a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b95a8033bc4f6e9da3d5abcec71af5, user_id=d7e95456a1814f1d893160cb9c9d0976, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.000771 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce119657-83e7-446e-bb45-80cab2005a07 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.004456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1296/2598] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:31.013030 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1303/2599] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:30 2022] PUT /identity/v3/projects/77b95a8033bc4f6e9da3d5abcec71af5/users/d7e95456a1814f1d893160cb9c9d0976/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:31.013232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ed76d-649b-48ac-ac2d-25bb30456e44 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.013450 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ed76d-649b-48ac-ac2d-25bb30456e44 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.013776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ed76d-649b-48ac-ac2d-25bb30456e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.024674 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.024908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.025163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.033897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-068ed76d-649b-48ac-ac2d-25bb30456e44 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:31.035106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-068ed76d-649b-48ac-ac2d-25bb30456e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:31.054254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] MFA Rules not processed for user `d7e95456a1814f1d893160cb9c9d0976`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:31.055382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:31.098920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1297/2600] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:31.103250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ed76d-649b-48ac-ac2d-25bb30456e44 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:31.113171 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ed76d-649b-48ac-ac2d-25bb30456e44 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:31.117935 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:31.118630 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.118915 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.119197 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.119933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:31.120239 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.121877 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.127411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1304/2601] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:31.136198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1298/2602] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:55:31.138994 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:31.141967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:31.145264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:31.155970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d20fa-e8db-4df1-aaa6-b2fd08d6adc8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:31.161125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:31.161785 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.162070 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.162348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.163042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.164980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.200841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489ac8f-29d8-4624-bef9-e9ecdfdd2827 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:31.204590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:31.205289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.205611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.205880 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.207809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:31.266409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.267928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.276976 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1305/2603] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:31.346250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1299/2604] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:31 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:31.347224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:31.366250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584a16d-deb9-4b8c-b66f-ce87c8ab3f80 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:31.439696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:31.440379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.440688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.441005 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.441709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.443934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.447987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:31.590914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1306/2605] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:31 2022] POST /identity/v3/users => generated 283 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:31.593945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:31.605206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52352dec-57f0-47b8-8554-26ddc0d09a64 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:31.609927 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:31.611174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.611462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.611745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.612502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.614178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.631799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1300/2606] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:31.634782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:31.645583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3a43-806f-442a-abb8-48f24a3bd84b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:31.650328 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fec5cc-0e16-4471-8017-82f064d4072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:31.650987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fec5cc-0e16-4471-8017-82f064d4072e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.651267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fec5cc-0e16-4471-8017-82f064d4072e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.651539 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fec5cc-0e16-4471-8017-82f064d4072e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.652260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fec5cc-0e16-4471-8017-82f064d4072e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.653897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fec5cc-0e16-4471-8017-82f064d4072e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.665089 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1307/2607] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:31.669296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:31.676298 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:31.680134 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705af07-93d9-47a1-ad2d-ebf49db92b48 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:31.684716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:31.685631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.685928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.686200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] PATH_INFO: `/v3/projects/a074120a798c4f858c952b8d5c76c275/users/2a0bd171e9254728bcd75f5e925c6252/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.701286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a074120a798c4f858c952b8d5c76c275, user_id=2a0bd171e9254728bcd75f5e925c6252, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:31.704532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:31.765931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:31.830583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1301/2608] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:31 2022] PUT /identity/v3/projects/a074120a798c4f858c952b8d5c76c275/users/2a0bd171e9254728bcd75f5e925c6252/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:31.841802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9041d3-b529-4214-aba4-67b027e92ca8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:31.842048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9041d3-b529-4214-aba4-67b027e92ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:31.842326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9041d3-b529-4214-aba4-67b027e92ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:31.871873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf9041d3-b529-4214-aba4-67b027e92ca8 None None] MFA Rules not processed for user `2a0bd171e9254728bcd75f5e925c6252`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:31.873163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf9041d3-b529-4214-aba4-67b027e92ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:31.940300 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1308/2609] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:31.945189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:32.007997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd729a-62c2-4dba-8dcf-45c6d6385d34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:32.012720 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:32.013386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.013670 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.013947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.014638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:32.017058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:32.130618 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1302/2610] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:31 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:32.132921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9041d3-b529-4214-aba4-67b027e92ca8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:32.144938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9041d3-b529-4214-aba4-67b027e92ca8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:32.205228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:32.206022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.206319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.206624 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.207338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:32.209388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:32.349386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1309/2611] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:32 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:32.353503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:32.364304 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab24cc-95be-4895-96cc-847c2abbc75e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:32.369051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:32.369731 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.370200 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.370452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.371556 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:32.374276 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:32.391600 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1303/2612] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:32.393454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:32.404062 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2bff-9d21-4e5c-8a65-440316d3694f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:32.408914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:32.409805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.410157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.410439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] PATH_INFO: `/v3/projects/5f8d92a97eb440ca98b80d03d12a38c0/users/0fc625bc2457421aaf20873688b24d86/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.425522 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8d92a97eb440ca98b80d03d12a38c0, user_id=0fc625bc2457421aaf20873688b24d86, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:32.428250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:32.601023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1310/2613] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:32 2022] PUT /identity/v3/projects/5f8d92a97eb440ca98b80d03d12a38c0/users/0fc625bc2457421aaf20873688b24d86/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:32.604957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:32.615504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa474-5bee-42c7-a7ce-64ab835fd912 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:32.681105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd13b3-f3c6-4b4a-802c-5aae850f787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:32.681781 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd13b3-f3c6-4b4a-802c-5aae850f787b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.682067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd13b3-f3c6-4b4a-802c-5aae850f787b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.682342 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd13b3-f3c6-4b4a-802c-5aae850f787b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.683074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd13b3-f3c6-4b4a-802c-5aae850f787b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:32.684675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd13b3-f3c6-4b4a-802c-5aae850f787b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:32.695695 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1304/2614] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:32.699839 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:32.710766 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581ad7-993b-432f-b94d-bfddb1feaf00 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:32.715740 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:32.716620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.716933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.717233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] PATH_INFO: `/v3/projects/5f8d92a97eb440ca98b80d03d12a38c0/users/0fc625bc2457421aaf20873688b24d86/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.719283 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8d92a97eb440ca98b80d03d12a38c0, user_id=0fc625bc2457421aaf20873688b24d86, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:32.722022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:32.728412 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:32.732897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1311/2615] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:32 2022] PUT /identity/v3/projects/5f8d92a97eb440ca98b80d03d12a38c0/users/0fc625bc2457421aaf20873688b24d86/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:32.752649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626925-4042-4099-a93c-808c158455f4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.752988 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626925-4042-4099-a93c-808c158455f4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.753279 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626925-4042-4099-a93c-808c158455f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.780079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b0626925-4042-4099-a93c-808c158455f4 None None] MFA Rules not processed for user `0fc625bc2457421aaf20873688b24d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:32.789189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b0626925-4042-4099-a93c-808c158455f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:32.825458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:32.854662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1305/2616] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:32.868308 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:32.882372 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:32.931546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b2181-3b64-4d90-bbfe-8bab4c69e0f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:32.935320 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:32.936037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:32.936321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:32.936596 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:32.948290 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:32.951072 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:32.961042 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1312/2617] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:33.305423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626925-4042-4099-a93c-808c158455f4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:33.319042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626925-4042-4099-a93c-808c158455f4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:33.322714 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:33.323365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:33.323649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:33.323949 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:33.383649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:33.384667 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:33.395384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1306/2618] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:33.428530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:33.438840 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328786c8-597f-4526-a682-fb7d7f8779b1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:33.447478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:33.447478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:33.447478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:33.447478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:33.447930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:33.512454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:33.513995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:33.525180 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1313/2619] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:33.891951 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:33.893381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:33.965479 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b467e-5322-4fcd-a12f-38abc49f2303 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:33.972032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:33.972032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:33.972032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:33.972032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:33.974141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:34.040874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:34.042333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:34.053206 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1307/2620] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:38.415207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:38.484114 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8469ba-456c-443f-9020-10e89b8d128c service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:38.489917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:38.490775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:38.491060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:38.491345 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] PATH_INFO: `/v3/users/2a0bd171e9254728bcd75f5e925c6252` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:38.492576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a0bd171e9254728bcd75f5e925c6252)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:38.494687 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:38.845014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:38.852067 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a0bd171e9254728bcd75f5e925c6252'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:38.865758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a0bd171e9254728bcd75f5e925c6252'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.063563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:39.075963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364056-2d71-4162-b150-0c8ad1896928 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:39.081160 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:39.082097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:39.082384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:39.082664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] PATH_INFO: `/v3/users/32dcff05551e4da7acf3753b5cebf041` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:39.084052 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=32dcff05551e4da7acf3753b5cebf041)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:39.086101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:39.108488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a0bd171e9254728bcd75f5e925c6252'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.111957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a0bd171e9254728bcd75f5e925c6252'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.111957 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f151b2b-ebe6-4a2a-b8f1-c5f9a98d0616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a0bd171e9254728bcd75f5e925c6252'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.113754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:39.124600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1314/2621] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:38 2022] DELETE /identity/v3/users/2a0bd171e9254728bcd75f5e925c6252 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:39.129216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:39.129518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:39.129809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:39.147129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:39.148401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:39.208166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1315/2622] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:39.218383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:39.219895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:39.286227 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e25818-63f7-4e54-a3f1-61310edd25fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:39.292805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa91c9f-d73c-4472-96ad-421be1abe00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:39.293507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa91c9f-d73c-4472-96ad-421be1abe00e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:39.293933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa91c9f-d73c-4472-96ad-421be1abe00e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:39.294220 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa91c9f-d73c-4472-96ad-421be1abe00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:39.311645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa91c9f-d73c-4472-96ad-421be1abe00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:39.313409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa91c9f-d73c-4472-96ad-421be1abe00e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:39.324862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1316/2623] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:39.461131 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32dcff05551e4da7acf3753b5cebf041'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.468869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32dcff05551e4da7acf3753b5cebf041'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.484844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32dcff05551e4da7acf3753b5cebf041'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.651980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32dcff05551e4da7acf3753b5cebf041'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.652601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32dcff05551e4da7acf3753b5cebf041'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:39.659775 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1308/2624] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:39 2022] DELETE /identity/v3/users/32dcff05551e4da7acf3753b5cebf041 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:39.665049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:39.665049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:39.665049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:39.698015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:39.698015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:39.752501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:39.763145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1317/2625] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:39.784312 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:39.786400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:39.860379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edf865-d920-415e-9737-6dfaa28079cf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:39.863747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:39.864419 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:39.864700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:39.865010 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:39.876309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:39.880962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:39.898655 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1309/2626] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:40.353651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:40.355360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:40.375158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99036db-ff63-42b6-81d4-abe4e6f9cee0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:40.381129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:40.390642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:40.391000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:40.391414 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] PATH_INFO: `/v3/projects/a074120a798c4f858c952b8d5c76c275` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:40.393364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a074120a798c4f858c952b8d5c76c275)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:40.395504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:40.529962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a074120a798c4f858c952b8d5c76c275', 'request_id': 'req-7548c80c-5432-4a70-8ac4-342ea5112a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:40.622961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a074120a798c4f858c952b8d5c76c275', 'request_id': 'req-7548c80c-5432-4a70-8ac4-342ea5112a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:40.632870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a074120a798c4f858c952b8d5c76c275', 'request_id': 'req-7548c80c-5432-4a70-8ac4-342ea5112a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:40.647902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] The token cache is being invalidate because project a074120a798c4f858c952b8d5c76c275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:40.648260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:40.649401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:40.651022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1318/2627] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:40 2022] DELETE /identity/v3/projects/a074120a798c4f858c952b8d5c76c275 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:40.655192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:40.666053 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1b24d-6955-435e-9922-d977f49ec71e service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:40.729060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:40.729782 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:40.730176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:40.730447 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] PATH_INFO: `/v3/users/0fc625bc2457421aaf20873688b24d86` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:40.731668 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc625bc2457421aaf20873688b24d86)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:40.733789 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:40.796437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:40.816011 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:40.866170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548c80c-5432-4a70-8ac4-342ea5112a5e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:40.878784 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:40.880208 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:40.880560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:40.880941 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] PATH_INFO: `/v3/projects/bfeea8b2c4f04089ad8caac0c0efd57d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:40.884350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfeea8b2c4f04089ad8caac0c0efd57d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:40.886447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:41.077654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:41.079926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc625bc2457421aaf20873688b24d86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.087080 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc625bc2457421aaf20873688b24d86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.100440 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc625bc2457421aaf20873688b24d86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.113704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfeea8b2c4f04089ad8caac0c0efd57d', 'request_id': 'req-fb414b23-46d0-4401-8705-043a72e3804e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.252876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc625bc2457421aaf20873688b24d86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.253616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc625bc2457421aaf20873688b24d86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.260381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1310/2628] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:40 2022] DELETE /identity/v3/users/0fc625bc2457421aaf20873688b24d86 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:41.381145 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfeea8b2c4f04089ad8caac0c0efd57d', 'request_id': 'req-fb414b23-46d0-4401-8705-043a72e3804e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.385433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfeea8b2c4f04089ad8caac0c0efd57d', 'request_id': 'req-fb414b23-46d0-4401-8705-043a72e3804e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.398470 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] The token cache is being invalidate because project bfeea8b2c4f04089ad8caac0c0efd57d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:41.398838 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.399508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.401196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1319/2629] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:40 2022] DELETE /identity/v3/projects/bfeea8b2c4f04089ad8caac0c0efd57d => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:41.405331 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:41.406827 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:41.475001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fefac4-5f9b-4d18-b9ce-7a80ba98b134 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:41.479907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:41.480847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:41.493410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:41.493410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] PATH_INFO: `/v3/users/d7e95456a1814f1d893160cb9c9d0976` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:41.493410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7e95456a1814f1d893160cb9c9d0976)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:41.493410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:41.820793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:41.822622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:41.830574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7e95456a1814f1d893160cb9c9d0976'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.852391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7e95456a1814f1d893160cb9c9d0976'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.866943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7e95456a1814f1d893160cb9c9d0976'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.900280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb414b23-46d0-4401-8705-043a72e3804e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:41.904870 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:41.905731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:41.906010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:41.906304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] PATH_INFO: `/v3/projects/5f8d92a97eb440ca98b80d03d12a38c0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:41.908459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8d92a97eb440ca98b80d03d12a38c0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:41.910507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:41.919652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:41.983443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7e95456a1814f1d893160cb9c9d0976'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.989141 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-99a4bb8a-9661-47da-aad9-20ef474280e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7e95456a1814f1d893160cb9c9d0976'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:41.992017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1311/2630] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:41 2022] DELETE /identity/v3/users/d7e95456a1814f1d893160cb9c9d0976 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:42.189990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8d92a97eb440ca98b80d03d12a38c0', 'request_id': 'req-c0627e3f-70f7-4106-a730-2b9e19b817ab'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:42.315195 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8d92a97eb440ca98b80d03d12a38c0', 'request_id': 'req-c0627e3f-70f7-4106-a730-2b9e19b817ab'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:42.327797 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8d92a97eb440ca98b80d03d12a38c0', 'request_id': 'req-c0627e3f-70f7-4106-a730-2b9e19b817ab'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:42.335827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] The token cache is being invalidate because project 5f8d92a97eb440ca98b80d03d12a38c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:42.336118 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:42.336835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0627e3f-70f7-4106-a730-2b9e19b817ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:42.338721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1320/2631] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:41 2022] DELETE /identity/v3/projects/5f8d92a97eb440ca98b80d03d12a38c0 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:42.343796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0a23-e0df-40b7-8740-fe5d83b0f655 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.344091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0a23-e0df-40b7-8740-fe5d83b0f655 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.344487 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0a23-e0df-40b7-8740-fe5d83b0f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.363723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0abd0a23-e0df-40b7-8740-fe5d83b0f655 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:42.369210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0abd0a23-e0df-40b7-8740-fe5d83b0f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:42.429127 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1312/2632] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:42.437358 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbca22-62d9-40dc-8b2e-bf194ef9866b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.437645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbca22-62d9-40dc-8b2e-bf194ef9866b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.437922 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbca22-62d9-40dc-8b2e-bf194ef9866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.459462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-50cbca22-62d9-40dc-8b2e-bf194ef9866b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:42.460738 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50cbca22-62d9-40dc-8b2e-bf194ef9866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:42.469329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1321/2633] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:42.474928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd0a23-e0df-40b7-8740-fe5d83b0f655 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:42.485562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd0a23-e0df-40b7-8740-fe5d83b0f655 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:42.490541 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:42.491270 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.491559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.491841 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.492629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:42.492971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:42.494620 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:42.509650 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1313/2634] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:55:42.515945 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbca22-62d9-40dc-8b2e-bf194ef9866b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:42.526634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbca22-62d9-40dc-8b2e-bf194ef9866b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:42.533581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:42.534270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.534554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.534830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.535530 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:42.537552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:42.716617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1322/2635] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:42 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:42.720803 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:42.731062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c97cc9-1c5d-4b77-b641-8242f323f32e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:42.787739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:42.788457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.788801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.789082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.789794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:42.791725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:42.848929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:42.850748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:42.864475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca18eee-9671-4e07-b983-292423b558e5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:42.869571 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:42.870480 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.870767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.871435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] PATH_INFO: `/v3/projects/77b95a8033bc4f6e9da3d5abcec71af5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.872382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b95a8033bc4f6e9da3d5abcec71af5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:42.874344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:42.915189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1314/2636] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:42 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:42.919060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:42.930783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c7b43-ff05-46be-96db-f7caad7ca2db admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:42.935463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:42.937379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.937379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.937379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.937379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:42.939579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:42.949934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1315/2637] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:42.954382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:42.964676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73559482-acf7-4c73-8c09-5372d0942cac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:42.970247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:42.971121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:42.971461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:42.971708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:42.972434 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:42.974047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:42.985000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1316/2638] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:42.989179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:42.998957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f705ebc-ab19-4cb8-a520-5c1fb158e390 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.003872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55caadbf-832f-432c-9a12-b116b13d7dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.004859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caadbf-832f-432c-9a12-b116b13d7dae admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.005167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caadbf-832f-432c-9a12-b116b13d7dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.005457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caadbf-832f-432c-9a12-b116b13d7dae admin admin] PATH_INFO: `/v3/projects/be9870640b02456ca92be03607935dcd/users/1d9201cb0c35431cb547ac0dbf15ba00/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.021913 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55caadbf-832f-432c-9a12-b116b13d7dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9870640b02456ca92be03607935dcd, user_id=1d9201cb0c35431cb547ac0dbf15ba00, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.024576 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55caadbf-832f-432c-9a12-b116b13d7dae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.138427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1317/2639] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:42 2022] PUT /identity/v3/projects/be9870640b02456ca92be03607935dcd/users/1d9201cb0c35431cb547ac0dbf15ba00/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:43.146887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b95a8033bc4f6e9da3d5abcec71af5', 'request_id': 'req-81c2a55b-db32-430e-b87b-51739f0011c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:43.149557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdbc1d-ba9a-4fb4-814e-3e50f6e9578a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.150025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdbc1d-ba9a-4fb4-814e-3e50f6e9578a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.150389 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdbc1d-ba9a-4fb4-814e-3e50f6e9578a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.185388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f9bdbc1d-ba9a-4fb4-814e-3e50f6e9578a None None] MFA Rules not processed for user `1d9201cb0c35431cb547ac0dbf15ba00`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:43.186687 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9bdbc1d-ba9a-4fb4-814e-3e50f6e9578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:43.229555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b95a8033bc4f6e9da3d5abcec71af5', 'request_id': 'req-81c2a55b-db32-430e-b87b-51739f0011c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:43.239326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b95a8033bc4f6e9da3d5abcec71af5', 'request_id': 'req-81c2a55b-db32-430e-b87b-51739f0011c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:43.253188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] The token cache is being invalidate because project 77b95a8033bc4f6e9da3d5abcec71af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:43.253575 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:43.254301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81c2a55b-db32-430e-b87b-51739f0011c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:43.256158 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1323/2640] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:42 2022] DELETE /identity/v3/projects/77b95a8033bc4f6e9da3d5abcec71af5 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:43.258435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1318/2641] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:43.261484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102199cc-f891-4ea0-976b-2b9229b9e50e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.262096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102199cc-f891-4ea0-976b-2b9229b9e50e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.262475 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102199cc-f891-4ea0-976b-2b9229b9e50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.262592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdbc1d-ba9a-4fb4-814e-3e50f6e9578a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.288589 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-102199cc-f891-4ea0-976b-2b9229b9e50e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:43.290402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-102199cc-f891-4ea0-976b-2b9229b9e50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:43.332553 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdbc1d-ba9a-4fb4-814e-3e50f6e9578a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.338919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.339617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.339907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.340189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.340934 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.342980 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.367779 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1324/2642] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:43.375554 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec9d8-fe0c-400e-ace0-2507321ee952 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.375829 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec9d8-fe0c-400e-ace0-2507321ee952 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.376110 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec9d8-fe0c-400e-ace0-2507321ee952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.396682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e52ec9d8-fe0c-400e-ace0-2507321ee952 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:43.396682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e52ec9d8-fe0c-400e-ace0-2507321ee952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:43.404091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1325/2643] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:43.409758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec9d8-fe0c-400e-ace0-2507321ee952 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.419057 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec9d8-fe0c-400e-ace0-2507321ee952 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.424564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.425261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.425543 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.425820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.426547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:43.426933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.428560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.441526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1326/2644] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:55:43.445708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.455876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed382b-3552-44aa-9459-c7ae934bc8c5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.460739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.461441 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.461728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.462009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.462964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.464919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.498824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1319/2645] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:43 2022] POST /identity/v3/projects => generated 387 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:43.502873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.504506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:43.567171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1327/2646] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:43 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:43.568140 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.571566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1366203-75a1-42e1-9d9d-f537df1832b0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.631740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.632413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.632701 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.633009 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.633712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.635090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f062c2-3778-4184-801c-bb5a1fa0559c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.635657 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.639826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.640500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.640805 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.641127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.642535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.644511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.765899 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1320/2647] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:43 2022] POST /identity/v3/users => generated 300 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:43.770170 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.780241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cc13a-8d71-4e4b-bd02-98cefd817168 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.785288 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.786012 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.786325 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.786640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.787395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.789130 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.800435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1321/2648] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:43.807412 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.808986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1328/2649] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:43 2022] POST /identity/v3/users => generated 284 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:43.812917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.818360 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c85fa6-d358-4eea-b942-a761622b0210 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.823322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.823521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184d1a6-3fb8-4aad-9613-533cc8694063 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.824208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.824491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.824824 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] PATH_INFO: `/v3/projects/acaf306268634ea19d1d5077e360dbcc/users/aa0c5a383d504da69162d6d866bf6012/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.829595 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.830251 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.830541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.830818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.831544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.833255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.840098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acaf306268634ea19d1d5077e360dbcc, user_id=aa0c5a383d504da69162d6d866bf6012, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.842770 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.843643 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1329/2650] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:43.848076 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.857967 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb8977-9595-40ab-ba2b-d1346d607453 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.863302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.863966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.864250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.864526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.865287 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.866955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.879317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1330/2651] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:43.883462 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.894393 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ec23e-980d-47e9-97e4-c4e222b493c7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:43.899329 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:43.900250 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:43.900545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:43.900855 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] PATH_INFO: `/v3/projects/4c9a2e91519744cd973e0f0939c92422/users/cfdc47394a83470b8b70ad8217408dc4/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:43.916127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9a2e91519744cd973e0f0939c92422, user_id=cfdc47394a83470b8b70ad8217408dc4, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:43.918853 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:43.932355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1322/2652] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:43 2022] PUT /identity/v3/projects/acaf306268634ea19d1d5077e360dbcc/users/aa0c5a383d504da69162d6d866bf6012/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:43.936297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:43.966064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:44.003203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e113e2-9c81-472c-b39d-10515efb7188 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:44.008000 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195fb69-a73f-4230-8917-181c8a286b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:44.008698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195fb69-a73f-4230-8917-181c8a286b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:44.009015 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195fb69-a73f-4230-8917-181c8a286b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:44.009287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195fb69-a73f-4230-8917-181c8a286b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:44.010069 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195fb69-a73f-4230-8917-181c8a286b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:44.011668 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195fb69-a73f-4230-8917-181c8a286b5a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:44.017975 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1331/2653] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:43 2022] PUT /identity/v3/projects/4c9a2e91519744cd973e0f0939c92422/users/cfdc47394a83470b8b70ad8217408dc4/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:44.023580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1323/2654] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:44.027315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:44.029852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:44.030144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:44.030423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:44.061387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] MFA Rules not processed for user `cfdc47394a83470b8b70ad8217408dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:44.062634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:44.093614 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea40e-3019-4b30-9950-3c083323c7c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:44.099149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17b66-fff7-43c6-a9aa-59b5c8698ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:44.100080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17b66-fff7-43c6-a9aa-59b5c8698ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:44.100368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17b66-fff7-43c6-a9aa-59b5c8698ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:44.100667 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17b66-fff7-43c6-a9aa-59b5c8698ba0 admin admin] PATH_INFO: `/v3/projects/acaf306268634ea19d1d5077e360dbcc/users/aa0c5a383d504da69162d6d866bf6012/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:44.102778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17b66-fff7-43c6-a9aa-59b5c8698ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acaf306268634ea19d1d5077e360dbcc, user_id=aa0c5a383d504da69162d6d866bf6012, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:44.105547 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17b66-fff7-43c6-a9aa-59b5c8698ba0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:44.117478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1332/2655] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:44 2022] PUT /identity/v3/projects/acaf306268634ea19d1d5077e360dbcc/users/aa0c5a383d504da69162d6d866bf6012/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:44.129421 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:44.129721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:44.130082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:44.131622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1324/2656] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:44.143365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:44.144755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:44.160384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] MFA Rules not processed for user `aa0c5a383d504da69162d6d866bf6012`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:44.161659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:44.173686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:44.207136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c3f84-8c29-4033-926f-9c4283e90ea5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:44.234673 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1333/2657] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:55:44.247226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:44.248523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:44.261976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e070ae-63d7-4496-af75-4c2948db5e61 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:44.262840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e070ae-63d7-4496-af75-4c2948db5e61 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:44.263384 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e070ae-63d7-4496-af75-4c2948db5e61 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:44.264135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e070ae-63d7-4496-af75-4c2948db5e61 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:44.313545 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d814d2-2090-4ca9-8904-727c0b46c11e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:44.317682 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:44.318405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:44.318754 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:44.319064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:44.330496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:44.333146 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:44.333852 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e070ae-63d7-4496-af75-4c2948db5e61 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:44.335433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e070ae-63d7-4496-af75-4c2948db5e61 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:44.343603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1334/2658] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:44.346945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1325/2659] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:44.377566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:44.472301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0102d-68ee-4bd9-a999-bb2bc1cc0101 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:44.480465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:44.480465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:44.480465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:44.480465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] PATH_INFO: `/v3/users/1d9201cb0c35431cb547ac0dbf15ba00` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:44.481619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9201cb0c35431cb547ac0dbf15ba00)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:44.484316 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:44.508968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:44.539683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:44.875160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9201cb0c35431cb547ac0dbf15ba00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:44.887850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9201cb0c35431cb547ac0dbf15ba00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:45.023347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:45.132101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9201cb0c35431cb547ac0dbf15ba00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:45.132732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9201cb0c35431cb547ac0dbf15ba00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:45.133485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9201cb0c35431cb547ac0dbf15ba00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:45.150507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1335/2660] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:44 2022] DELETE /identity/v3/users/1d9201cb0c35431cb547ac0dbf15ba00 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:45.155969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:45.156258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:45.156535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:45.177726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:45.178961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:45.236772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:45.265332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1326/2661] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:45.273003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:45.274447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:45.337787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb54cc9-028a-4751-b1ba-87d86f91f589 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:45.341520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:45.342253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:45.342542 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:45.342820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:45.353819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:45.355301 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:45.365698 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1336/2662] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:45.788681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:46.038443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:46.038443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:46.046125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afcf6c-3f61-49b0-a59e-70404b92b620 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:46.052422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:46.053218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:46.053691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:46.054890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:46.054890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] PATH_INFO: `/v3/projects/be9870640b02456ca92be03607935dcd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:46.055358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9870640b02456ca92be03607935dcd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:46.057395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:46.062425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297fcc0-dcc8-42c6-b356-482e7475a238 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:46.067160 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:46.067914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:46.068194 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:46.068473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:46.070380 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:46.084049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:46.085558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:46.097259 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1337/2663] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:55:46.210354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9870640b02456ca92be03607935dcd', 'request_id': 'req-c0324f07-7bf2-43dd-81b7-3f7985a9916f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:46.217631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9870640b02456ca92be03607935dcd', 'request_id': 'req-c0324f07-7bf2-43dd-81b7-3f7985a9916f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:46.382942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9870640b02456ca92be03607935dcd', 'request_id': 'req-c0324f07-7bf2-43dd-81b7-3f7985a9916f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:46.396433 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] The token cache is being invalidate because project be9870640b02456ca92be03607935dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:46.396736 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:46.397483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:46.399386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1327/2664] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:46 2022] DELETE /identity/v3/projects/be9870640b02456ca92be03607935dcd => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:46.403533 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:46.414339 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9979ea-0e71-4464-97d3-af059436f734 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:46.479454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:46.480267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:46.480551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:46.480866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] PATH_INFO: `/v3/users/aa0c5a383d504da69162d6d866bf6012` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:46.482108 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0c5a383d504da69162d6d866bf6012)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:46.484161 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:46.651857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:46.790607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0c5a383d504da69162d6d866bf6012'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:46.828722 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0c5a383d504da69162d6d866bf6012'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:47.172090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0c5a383d504da69162d6d866bf6012'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:47.173048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0c5a383d504da69162d6d866bf6012'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:47.173990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f7eb9e7-a031-4225-a624-63ea676eba14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0c5a383d504da69162d6d866bf6012'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:47.196486 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1338/2665] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:46 2022] DELETE /identity/v3/users/aa0c5a383d504da69162d6d866bf6012 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:47.816728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:47.818727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:47.840059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0324f07-7bf2-43dd-81b7-3f7985a9916f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:47.848203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:47.912971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:47.919470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:47.919470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:47.919470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] PATH_INFO: `/v3/projects/acaf306268634ea19d1d5077e360dbcc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:47.926491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acaf306268634ea19d1d5077e360dbcc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:47.926491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:48.089820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acaf306268634ea19d1d5077e360dbcc', 'request_id': 'req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:48.098066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acaf306268634ea19d1d5077e360dbcc', 'request_id': 'req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:48.214762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acaf306268634ea19d1d5077e360dbcc', 'request_id': 'req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:48.246654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] The token cache is being invalidate because project acaf306268634ea19d1d5077e360dbcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:48.246968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:48.247637 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01f41ec5-9a2b-4ce5-94c2-997f65eddc28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:48.252743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1328/2666] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:47 2022] DELETE /identity/v3/projects/acaf306268634ea19d1d5077e360dbcc => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:48.257748 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9462fb-dbe1-4342-9a55-a5425d5f67f4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.258732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9462fb-dbe1-4342-9a55-a5425d5f67f4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.259082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9462fb-dbe1-4342-9a55-a5425d5f67f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.281152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3d9462fb-dbe1-4342-9a55-a5425d5f67f4 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:48.283213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d9462fb-dbe1-4342-9a55-a5425d5f67f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:48.350214 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1339/2667] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:48.358411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed3f45-4886-4eb3-80d7-a7722999f283 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.358683 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed3f45-4886-4eb3-80d7-a7722999f283 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.358965 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed3f45-4886-4eb3-80d7-a7722999f283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.384234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f0ed3f45-4886-4eb3-80d7-a7722999f283 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:48.385691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0ed3f45-4886-4eb3-80d7-a7722999f283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:48.395099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1329/2668] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:48.399561 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9462fb-dbe1-4342-9a55-a5425d5f67f4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:48.410935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9462fb-dbe1-4342-9a55-a5425d5f67f4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:48.419858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:48.420598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.420917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.421209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.422109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:55:48.422993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:48.424594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:48.438298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1340/2669] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:55:48.442560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed3f45-4886-4eb3-80d7-a7722999f283 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:48.456651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed3f45-4886-4eb3-80d7-a7722999f283 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:48.462045 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:48.463154 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.463536 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.463927 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.464804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:48.467086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:48.577371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1330/2670] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:48 2022] POST /identity/v3/projects => generated 397 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:55:48.582306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:48.593515 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38bbc-1eb6-4e96-b613-725c50dc6c29 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:48.666256 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:48.666995 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.667286 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.668078 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.668295 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:48.670371 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:48.790833 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1341/2671] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:48 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:48.794820 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:48.813801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5804b-c659-460a-9ecc-94644cf03b03 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:48.823082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:48.823780 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.824071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.824353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.825174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:48.828230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:48.840461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1331/2672] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:48.847003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:48.856531 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6886aa-fda3-4cc7-83c9-26f432054c97 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:48.862477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4daa09-12c7-4a2d-bc8e-80829962efc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:48.863170 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4daa09-12c7-4a2d-bc8e-80829962efc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.863454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4daa09-12c7-4a2d-bc8e-80829962efc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.863728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4daa09-12c7-4a2d-bc8e-80829962efc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.864512 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4daa09-12c7-4a2d-bc8e-80829962efc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:48.866202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4daa09-12c7-4a2d-bc8e-80829962efc4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:48.877504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1342/2673] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:48.881716 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:48.892398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be77cf9-dc49-43d8-a089-820633e2d78e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:48.901336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:48.902269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.902555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.902833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] PATH_INFO: `/v3/projects/f196383d58c44f089c667521500ad098/users/aa805dc9ab1544d0a6f49182021ec7d0/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.923638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f196383d58c44f089c667521500ad098, user_id=aa805dc9ab1544d0a6f49182021ec7d0, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:48.927136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:49.035151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1332/2674] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:48 2022] PUT /identity/v3/projects/f196383d58c44f089c667521500ad098/users/aa805dc9ab1544d0a6f49182021ec7d0/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:49.045799 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c63b33-0386-4c13-94d4-3c2efc02f2ce None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:49.046098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c63b33-0386-4c13-94d4-3c2efc02f2ce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:49.046382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c63b33-0386-4c13-94d4-3c2efc02f2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:49.065079 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:49.073772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-13c63b33-0386-4c13-94d4-3c2efc02f2ce None None] MFA Rules not processed for user `aa805dc9ab1544d0a6f49182021ec7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:49.076060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13c63b33-0386-4c13-94d4-3c2efc02f2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:49.146723 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1343/2675] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:49.151319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:49.223681 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39415ef0-ded3-4744-bc7a-07a598810d1d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:49.229609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:49.230341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:49.230666 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:49.230950 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:49.231834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:49.233978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:49.424819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1333/2676] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:55:49 2022] POST /identity/v3/projects => generated 401 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:49.431013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c63b33-0386-4c13-94d4-3c2efc02f2ce None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:49.443666 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c63b33-0386-4c13-94d4-3c2efc02f2ce None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:49.506123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:49.506874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:49.507163 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:49.507445 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:49.508156 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:49.510200 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:49.670132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1344/2677] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:55:49 2022] POST /identity/v3/users => generated 307 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:55:49.674308 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:49.685320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a4db4-897e-4f33-a4fb-bee7080a4792 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:49.690822 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:49.691625 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:49.691916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:49.692193 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:49.693143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:49.694723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:49.712459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1334/2678] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:55:49.716920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:49.728606 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8d0f3-de29-4804-a3b3-df5052ff1b27 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:49.736082 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:49.737377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:49.737861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:49.738349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] PATH_INFO: `/v3/projects/93cdb4712e694d1183fb6fbd7af2cc6e/users/50f7993bdac4455fbe84d07d7a22def9/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:49.749256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:49.756386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] RBAC: Authorizing `identity:create_grant(project_id=93cdb4712e694d1183fb6fbd7af2cc6e, user_id=50f7993bdac4455fbe84d07d7a22def9, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:49.759223 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:49.846558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1345/2679] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:49 2022] PUT /identity/v3/projects/93cdb4712e694d1183fb6fbd7af2cc6e/users/50f7993bdac4455fbe84d07d7a22def9/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:49.848185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:49.863516 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc806152-8ba4-49fe-bd53-ab06bcb88c4b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:49.932638 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3f2bd-abbc-4695-833b-af3444c24302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:49.933391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3f2bd-abbc-4695-833b-af3444c24302 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:49.933764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3f2bd-abbc-4695-833b-af3444c24302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:49.934111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3f2bd-abbc-4695-833b-af3444c24302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:49.935076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3f2bd-abbc-4695-833b-af3444c24302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:49.936941 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3f2bd-abbc-4695-833b-af3444c24302 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:49.951452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1335/2680] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:55:49.953023 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:49.965028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647216-efb7-42ca-a9b6-ead052badafb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:49.971338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:49.972326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:49.972686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:49.973102 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] PATH_INFO: `/v3/projects/93cdb4712e694d1183fb6fbd7af2cc6e/users/50f7993bdac4455fbe84d07d7a22def9/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:49.975415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93cdb4712e694d1183fb6fbd7af2cc6e, user_id=50f7993bdac4455fbe84d07d7a22def9, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:49.978759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:50.001948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1346/2681] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:55:49 2022] PUT /identity/v3/projects/93cdb4712e694d1183fb6fbd7af2cc6e/users/50f7993bdac4455fbe84d07d7a22def9/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:50.010679 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd448f1-b274-4306-9f55-f670e6d1f8ab None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:50.014499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd448f1-b274-4306-9f55-f670e6d1f8ab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:50.014499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd448f1-b274-4306-9f55-f670e6d1f8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:50.050641 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ccd448f1-b274-4306-9f55-f670e6d1f8ab None None] MFA Rules not processed for user `50f7993bdac4455fbe84d07d7a22def9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:50.052116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ccd448f1-b274-4306-9f55-f670e6d1f8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:50.146759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1336/2682] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:50.158228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:50.160026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:50.244086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e8cfb-8b95-4a21-bdb0-bbedeca36ad9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:50.249216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:50.256942 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:50.257569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:50.257728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:50.268987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:50.270521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:50.283335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1347/2683] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:50.320432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:50.547300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd448f1-b274-4306-9f55-f670e6d1f8ab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:50.557942 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd448f1-b274-4306-9f55-f670e6d1f8ab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:50.562423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:50.563138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:50.563424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:50.563700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:50.616900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:50.648264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:50.650079 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:50.661520 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1337/2684] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:54.794212 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:54.795702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:54.872054 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a921d-06cb-4192-9092-4847fffc7d76 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:54.878394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:54.879056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:54.879343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:54.879619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:54.881511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:54.894106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:54.895721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:54.906812 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1348/2685] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:54.960703 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:54.962663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:55.035445 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c09f1f-8955-43df-9265-028308a74e81 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:55.040017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:55.040997 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:55.041287 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:55.041563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] PATH_INFO: `/v3/users/926d4c4971f1425b98aa5828f802fb21` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:55.042808 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=926d4c4971f1425b98aa5828f802fb21)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:55.044760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:55.265030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926d4c4971f1425b98aa5828f802fb21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:55.272653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926d4c4971f1425b98aa5828f802fb21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:55.285711 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926d4c4971f1425b98aa5828f802fb21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:55.433912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926d4c4971f1425b98aa5828f802fb21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:55.434635 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926d4c4971f1425b98aa5828f802fb21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:55.442544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1338/2686] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:54 2022] DELETE /identity/v3/users/926d4c4971f1425b98aa5828f802fb21 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:55.722513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:55.736485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e7284-7eb3-42cd-8149-e4919bbf0161 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:55.806028 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:55.806908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:55.807206 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:55.809936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] PATH_INFO: `/v3/projects/7f36b12884cc423988d28fdcbdb1d8f9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:55.811341 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f36b12884cc423988d28fdcbdb1d8f9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:55.813518 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:55.960758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f36b12884cc423988d28fdcbdb1d8f9', 'request_id': 'req-b5312ccf-fa9b-4b61-989c-a25a42b808ca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.091226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f36b12884cc423988d28fdcbdb1d8f9', 'request_id': 'req-b5312ccf-fa9b-4b61-989c-a25a42b808ca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.099338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f36b12884cc423988d28fdcbdb1d8f9', 'request_id': 'req-b5312ccf-fa9b-4b61-989c-a25a42b808ca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.110349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] The token cache is being invalidate because project 7f36b12884cc423988d28fdcbdb1d8f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:56.110641 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.111330 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.113012 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1349/2687] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:55 2022] DELETE /identity/v3/projects/7f36b12884cc423988d28fdcbdb1d8f9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:56.117224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:56.118776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:56.133265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22bd5-106a-4d9c-b209-28675779e789 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:56.180729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:56.193712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:56.194602 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:56.194887 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:56.195185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] PATH_INFO: `/v3/users/e8543b358034424b89084f01be07cabe` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:56.196427 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8543b358034424b89084f01be07cabe)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:56.198422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:56.414157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8543b358034424b89084f01be07cabe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.421168 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8543b358034424b89084f01be07cabe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.433456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8543b358034424b89084f01be07cabe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.519670 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8543b358034424b89084f01be07cabe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.520315 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8543b358034424b89084f01be07cabe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:56.528212 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1339/2688] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:56 2022] DELETE /identity/v3/users/e8543b358034424b89084f01be07cabe => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:56.909883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:56.911622 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:56.927397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5312ccf-fa9b-4b61-989c-a25a42b808ca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:56.943484 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:57.015884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:57.017656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:57.017656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:57.017656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] PATH_INFO: `/v3/projects/156f859134cf41cf985d337c94c6ed77` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:57.018633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156f859134cf41cf985d337c94c6ed77)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:57.020656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:57.148125 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156f859134cf41cf985d337c94c6ed77', 'request_id': 'req-714157fc-e52a-4a2a-a9e9-8bf2629499f5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:57.291264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156f859134cf41cf985d337c94c6ed77', 'request_id': 'req-714157fc-e52a-4a2a-a9e9-8bf2629499f5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:57.300520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156f859134cf41cf985d337c94c6ed77', 'request_id': 'req-714157fc-e52a-4a2a-a9e9-8bf2629499f5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:57.319003 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] The token cache is being invalidate because project 156f859134cf41cf985d337c94c6ed77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:55:57.321416 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:57.322487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-714157fc-e52a-4a2a-a9e9-8bf2629499f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:57.324686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1350/2689] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:55:56 2022] DELETE /identity/v3/projects/156f859134cf41cf985d337c94c6ed77 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:55:59.338873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:59.340414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:59.354704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e2c96-9fa5-4a9a-a8ae-b005ec66dd09 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:59.450604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:59.451503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:59.451794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:59.452073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] PATH_INFO: `/v3/users/aa805dc9ab1544d0a6f49182021ec7d0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:59.454459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa805dc9ab1544d0a6f49182021ec7d0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:59.456321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:59.775910 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa805dc9ab1544d0a6f49182021ec7d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:59.784136 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa805dc9ab1544d0a6f49182021ec7d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:59.794957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:59.798250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa805dc9ab1544d0a6f49182021ec7d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:59.919530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa805dc9ab1544d0a6f49182021ec7d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:59.920375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa805dc9ab1544d0a6f49182021ec7d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:55:59.930067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1340/2690] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:55:59 2022] DELETE /identity/v3/users/aa805dc9ab1544d0a6f49182021ec7d0 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:55:59.937607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3fd28-176e-4727-b1bf-0e41b09395dc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:59.937607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3fd28-176e-4727-b1bf-0e41b09395dc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:59.937607 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3fd28-176e-4727-b1bf-0e41b09395dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:59.956335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fbc3fd28-176e-4727-b1bf-0e41b09395dc None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:59.957841 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fbc3fd28-176e-4727-b1bf-0e41b09395dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:00.037635 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1351/2691] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:00.048203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:00.050158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:00.120508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4594ec-022d-41e9-ad7c-872dc465f5c4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:00.126439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:00.127256 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:00.127678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:00.128032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:00.138559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:00.140107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:00.152985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1341/2692] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:00.580806 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3fd28-176e-4727-b1bf-0e41b09395dc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:00.591897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3fd28-176e-4727-b1bf-0e41b09395dc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:00.596338 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:00.597036 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:00.597315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:00.597584 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:00.599390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:00.675277 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:00.710352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:00.714013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:00.729442 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1352/2693] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:00.916073 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:00.917806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:00.934890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e926c25-bbae-4dcd-af5b-8da5cdd3c061 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:00.944089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:00.945032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:00.945408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:00.945694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] PATH_INFO: `/v3/projects/f196383d58c44f089c667521500ad098` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:00.946974 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f196383d58c44f089c667521500ad098)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:00.948947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:01.104060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f196383d58c44f089c667521500ad098', 'request_id': 'req-ce30adca-66a5-45f4-beac-f58abbdec25a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.115870 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f196383d58c44f089c667521500ad098', 'request_id': 'req-ce30adca-66a5-45f4-beac-f58abbdec25a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.239704 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f196383d58c44f089c667521500ad098', 'request_id': 'req-ce30adca-66a5-45f4-beac-f58abbdec25a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.257285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] The token cache is being invalidate because project f196383d58c44f089c667521500ad098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:01.257706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.258379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.260577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1342/2694] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:00 2022] DELETE /identity/v3/projects/f196383d58c44f089c667521500ad098 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:01.267240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:01.279786 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a91b83-3656-4189-9837-1c25867bded8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:01.372046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:01.372972 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:01.373267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:01.373544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] PATH_INFO: `/v3/users/50f7993bdac4455fbe84d07d7a22def9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:01.374785 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f7993bdac4455fbe84d07d7a22def9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:01.376937 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:01.631080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f7993bdac4455fbe84d07d7a22def9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.645560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f7993bdac4455fbe84d07d7a22def9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.748106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f7993bdac4455fbe84d07d7a22def9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.748926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f7993bdac4455fbe84d07d7a22def9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.749718 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dec7b268-f7a2-4af0-a3e6-e7b625f4a5c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f7993bdac4455fbe84d07d7a22def9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:01.771205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1353/2695] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:01 2022] DELETE /identity/v3/users/50f7993bdac4455fbe84d07d7a22def9 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:02.720532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:02.722155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:02.741718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30adca-66a5-45f4-beac-f58abbdec25a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:02.830920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:02.832672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:02.833062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:02.833472 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] PATH_INFO: `/v3/projects/93cdb4712e694d1183fb6fbd7af2cc6e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:02.838873 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93cdb4712e694d1183fb6fbd7af2cc6e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:02.841157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:02.864464 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:03.030712 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93cdb4712e694d1183fb6fbd7af2cc6e', 'request_id': 'req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:03.039251 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93cdb4712e694d1183fb6fbd7af2cc6e', 'request_id': 'req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:03.182859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93cdb4712e694d1183fb6fbd7af2cc6e', 'request_id': 'req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:03.198452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] The token cache is being invalidate because project 93cdb4712e694d1183fb6fbd7af2cc6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:03.198748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:03.200940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-29f2ff8e-2bb5-4c94-b3d5-9c49ab4ad011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:03.201405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1343/2696] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:02 2022] DELETE /identity/v3/projects/93cdb4712e694d1183fb6fbd7af2cc6e => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:03.206966 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b56b7-43f7-4156-92c9-1421f75d28b7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:03.207608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b56b7-43f7-4156-92c9-1421f75d28b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:03.207608 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b56b7-43f7-4156-92c9-1421f75d28b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:03.227334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3d8b56b7-43f7-4156-92c9-1421f75d28b7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:03.228699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d8b56b7-43f7-4156-92c9-1421f75d28b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:03.302564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:03.313686 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1354/2697] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:03.321762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66989bd3-97ff-44ae-b468-0953ca88f963 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:03.322054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66989bd3-97ff-44ae-b468-0953ca88f963 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:03.322338 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66989bd3-97ff-44ae-b468-0953ca88f963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:03.345354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66989bd3-97ff-44ae-b468-0953ca88f963 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:03.346893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66989bd3-97ff-44ae-b468-0953ca88f963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:03.358463 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1344/2698] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:03.366132 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b56b7-43f7-4156-92c9-1421f75d28b7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:03.386564 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b56b7-43f7-4156-92c9-1421f75d28b7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:03.394645 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:03.395775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:03.396270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:03.396749 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:03.398228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:03.398767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:03.401737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:03.427379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1355/2699] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:56:03.428955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66989bd3-97ff-44ae-b468-0953ca88f963 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:03.440617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66989bd3-97ff-44ae-b468-0953ca88f963 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:03.446056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:03.446863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:03.447218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:03.447639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:03.448431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:03.450530 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:03.660398 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1345/2700] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:03 2022] POST /identity/v3/projects => generated 365 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:03.664256 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:03.688955 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083b12d-8b48-4e54-bae4-be37b3aeea48 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:03.751280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:03.753275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:03.832931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:03.834428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:03.834907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:03.835440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:03.836697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:03.840671 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:03.858682 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701859a8-7284-454c-8790-9ed9df8636f8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:03.871634 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:03.873869 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:03.876985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:03.876985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] PATH_INFO: `/v3/users/a346c9634e6144e982812662451ba6e5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:03.877975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a346c9634e6144e982812662451ba6e5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:03.881993 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:03.955636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1356/2701] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:03 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:03.959696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:03.971793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7bc25-6331-47f4-a10d-d45b3beaa8cc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:03.980731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:03.980731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:03.980731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:03.980731 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:03.981137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:03.981778 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:03.994157 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1357/2702] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:56:03.998126 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:04.009694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75b2b-3bdf-412f-adbf-69b1a0c09e50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:04.019579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:04.019579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.019579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.019579 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.019939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:04.019939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:04.031355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1358/2703] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:56:04.036002 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:04.047793 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f10e-cc91-4888-84ca-3ef798e5321e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:04.052714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef793aa9-0a30-48c6-984a-f56a9b6ba224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:04.054750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793aa9-0a30-48c6-984a-f56a9b6ba224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.054750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793aa9-0a30-48c6-984a-f56a9b6ba224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.054750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793aa9-0a30-48c6-984a-f56a9b6ba224 admin admin] PATH_INFO: `/v3/projects/ecccb4837a3f428bb925dc833d765e5a/users/f50ef229e7b847c39cf3808a3883a023/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.070702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef793aa9-0a30-48c6-984a-f56a9b6ba224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecccb4837a3f428bb925dc833d765e5a, user_id=f50ef229e7b847c39cf3808a3883a023, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:04.073800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef793aa9-0a30-48c6-984a-f56a9b6ba224 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:04.089348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a346c9634e6144e982812662451ba6e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:04.096958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a346c9634e6144e982812662451ba6e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:04.109984 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a346c9634e6144e982812662451ba6e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:04.186373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1359/2704] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:04 2022] PUT /identity/v3/projects/ecccb4837a3f428bb925dc833d765e5a/users/f50ef229e7b847c39cf3808a3883a023/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:04.197936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.198229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.198485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.211570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a346c9634e6144e982812662451ba6e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:04.212420 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa7abc42-6e5d-45c6-ab0e-fb0b8a7c09f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a346c9634e6144e982812662451ba6e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:04.226077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1346/2705] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:03 2022] DELETE /identity/v3/users/a346c9634e6144e982812662451ba6e5 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:04.230508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.230890 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.231264 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.258214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:04.259664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] MFA Rules not processed for user `f50ef229e7b847c39cf3808a3883a023`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:04.260153 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:04.262428 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:04.294244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:04.373240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1360/2706] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:04.376254 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1347/2707] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:04.382621 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:04.384861 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:04.386087 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:04.388539 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:04.400025 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d014d-70b2-4010-b360-44c46600d2e0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:04.407184 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:04.408129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.408504 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.408902 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.409745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:04.412071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:04.435015 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:04.469129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66f28-7a2c-4e35-ad4e-e0cf67de01be None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:04.474940 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:04.475652 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.475989 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.476274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.487271 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:04.488738 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:04.504347 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1348/2708] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:04.562985 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1361/2709] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:04 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:04.568291 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:04.580198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91046a55-1370-47a5-b9ff-2c1c49c019d1 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:04.672719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:04.673663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.674025 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.674380 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.675218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:04.677565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:04.858960 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1349/2710] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:04 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:04.863859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:04.876923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6b5c9-6fd4-49f3-9daa-65f667484964 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:04.882177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:04.882899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.883215 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.883437 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.884186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:04.885885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:04.899893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1362/2711] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:04.904249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:04.917114 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aa1a1-7539-4446-b377-b52964270ac1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:04.926373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:04.927249 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:04.927696 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:04.928129 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] PATH_INFO: `/v3/projects/05eb7bc3c799461a8a334e96d1b95008/users/e9a81eb7fe95465da488761b530db8d4/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:04.949868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05eb7bc3c799461a8a334e96d1b95008, user_id=e9a81eb7fe95465da488761b530db8d4, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:04.952631 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:05.054109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:05.056149 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:05.075359 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c065d4f-2fe1-402c-b19a-dc6841c8d140 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:05.076260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1350/2712] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:04 2022] PUT /identity/v3/projects/05eb7bc3c799461a8a334e96d1b95008/users/e9a81eb7fe95465da488761b530db8d4/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:05.081567 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:05.091283 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:05.160993 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:05.161898 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:05.162229 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:05.162509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] PATH_INFO: `/v3/projects/abbff9e15bad46c4a13054f9a8e7c758` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:05.163930 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abbff9e15bad46c4a13054f9a8e7c758)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:05.165952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:05.182381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9f963-ae3d-4dc8-bde7-56c8f22eb6d3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:05.187845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:05.188527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:05.189083 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:05.189364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:05.190109 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:05.191759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:05.202395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1351/2713] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:56:05.207518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:05.218303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374657-653c-4f1d-9d5c-fb7e18eed26f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:05.223814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11146-6a0b-4461-9225-972b012ebef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:05.224715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11146-6a0b-4461-9225-972b012ebef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:05.225039 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11146-6a0b-4461-9225-972b012ebef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:05.225320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11146-6a0b-4461-9225-972b012ebef9 admin admin] PATH_INFO: `/v3/projects/05eb7bc3c799461a8a334e96d1b95008/users/e9a81eb7fe95465da488761b530db8d4/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:05.227428 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11146-6a0b-4461-9225-972b012ebef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05eb7bc3c799461a8a334e96d1b95008, user_id=e9a81eb7fe95465da488761b530db8d4, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:05.230123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11146-6a0b-4461-9225-972b012ebef9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:05.241806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1352/2714] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:05 2022] PUT /identity/v3/projects/05eb7bc3c799461a8a334e96d1b95008/users/e9a81eb7fe95465da488761b530db8d4/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:05.253266 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:05.253563 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:05.253845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:05.282097 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] MFA Rules not processed for user `e9a81eb7fe95465da488761b530db8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:05.283483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:05.331275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abbff9e15bad46c4a13054f9a8e7c758', 'request_id': 'req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.363651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1353/2715] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:05.376319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:05.378123 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:05.438587 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160bc77-d03b-463a-af25-c7cc4b5da8c5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:05.491298 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abbff9e15bad46c4a13054f9a8e7c758', 'request_id': 'req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.493332 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:05.494774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:05.495054 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:05.495334 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:05.499438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abbff9e15bad46c4a13054f9a8e7c758', 'request_id': 'req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.513943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] The token cache is being invalidate because project abbff9e15bad46c4a13054f9a8e7c758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:05.514234 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.514946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.516552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1363/2716] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:05 2022] DELETE /identity/v3/projects/abbff9e15bad46c4a13054f9a8e7c758 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:05.521153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:05.522171 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:05.584797 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:05.586551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:05.599702 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1354/2717] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:05.603285 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab2d63-7d1e-455e-a4b9-1a3cd3bb86a7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:05.608929 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:05.610459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:05.616006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:05.616006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] PATH_INFO: `/v3/users/73776a6bd97743bfb392ba4fbfc6e224` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:05.616006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=73776a6bd97743bfb392ba4fbfc6e224)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:05.616006 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:05.635628 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:05.702628 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:05.861181 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73776a6bd97743bfb392ba4fbfc6e224'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.873430 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73776a6bd97743bfb392ba4fbfc6e224'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.903055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:05.912767 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d39b2c-c97f-476d-abe1-97a838422ace service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:05.919293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf1176-cd0c-420e-a703-b88cceed20f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:05.919998 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf1176-cd0c-420e-a703-b88cceed20f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:05.920285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf1176-cd0c-420e-a703-b88cceed20f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:05.920562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf1176-cd0c-420e-a703-b88cceed20f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:05.923381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1dcf1176-cd0c-420e-a703-b88cceed20f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:05.965677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73776a6bd97743bfb392ba4fbfc6e224'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.966322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73776a6bd97743bfb392ba4fbfc6e224'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.968333 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73776a6bd97743bfb392ba4fbfc6e224'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:05.985816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1364/2718] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:05 2022] DELETE /identity/v3/users/73776a6bd97743bfb392ba4fbfc6e224 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:06.013329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf1176-cd0c-420e-a703-b88cceed20f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:06.014815 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf1176-cd0c-420e-a703-b88cceed20f0 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:06.032265 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1355/2719] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:06.528707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:06.530364 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:06.621857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db7a84-6e9d-4342-8cee-032aae758cba admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:06.628014 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:06.628936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:06.629217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:06.629487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] PATH_INFO: `/v3/projects/7b8b62003c0a4a45947d0537cd7d386c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:06.630775 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b8b62003c0a4a45947d0537cd7d386c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:06.632862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:06.794388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b8b62003c0a4a45947d0537cd7d386c', 'request_id': 'req-f6b44885-7300-4e1b-808e-b575ff903b59'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:06.982594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b8b62003c0a4a45947d0537cd7d386c', 'request_id': 'req-f6b44885-7300-4e1b-808e-b575ff903b59'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:06.991438 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b8b62003c0a4a45947d0537cd7d386c', 'request_id': 'req-f6b44885-7300-4e1b-808e-b575ff903b59'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:07.006107 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] The token cache is being invalidate because project 7b8b62003c0a4a45947d0537cd7d386c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:07.006402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:07.007081 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f6b44885-7300-4e1b-808e-b575ff903b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:07.008988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1365/2720] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:06 2022] DELETE /identity/v3/projects/7b8b62003c0a4a45947d0537cd7d386c => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:07.023529 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947343c-8879-4f1b-99e6-fc5c25fa935f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.024275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947343c-8879-4f1b-99e6-fc5c25fa935f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.024275 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947343c-8879-4f1b-99e6-fc5c25fa935f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.046057 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8947343c-8879-4f1b-99e6-fc5c25fa935f None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:07.048956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8947343c-8879-4f1b-99e6-fc5c25fa935f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:07.119076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1356/2721] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:07.127762 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc89c8-bbb2-4020-97ec-fcd4e247710c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.128049 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc89c8-bbb2-4020-97ec-fcd4e247710c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.128326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc89c8-bbb2-4020-97ec-fcd4e247710c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.147461 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-91fc89c8-bbb2-4020-97ec-fcd4e247710c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:07.148692 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-91fc89c8-bbb2-4020-97ec-fcd4e247710c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:07.158377 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1366/2722] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:07.163155 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947343c-8879-4f1b-99e6-fc5c25fa935f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:07.174623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947343c-8879-4f1b-99e6-fc5c25fa935f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:07.180056 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:07.180847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.181124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.181586 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.182353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:07.182958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:07.184832 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:07.200274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1357/2723] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:56:07.222851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc89c8-bbb2-4020-97ec-fcd4e247710c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:07.233471 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc89c8-bbb2-4020-97ec-fcd4e247710c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:07.245020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:07.245739 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.246033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.246307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.247024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:07.248999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:07.265270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:07.413948 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1367/2724] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:07 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:07.418547 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:07.424856 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:07.430333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5ae37-11b8-488a-ba45-4341a1105a97 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:07.488564 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:07.489375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.489658 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.489943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.490778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:07.492686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:07.635443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1358/2725] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:07 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:07.639909 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:07.650962 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02840451-a344-495d-9df8-3489ac786cc0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:07.657239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:07.658334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.658802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.659261 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.660521 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:07.663422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:07.680209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1368/2726] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:07.704435 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:07.716507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20d104-696b-43cd-a596-9ef11eb57dfe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:07.722371 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c49ab-e621-4b9e-85b9-1f02daa2868b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:07.723466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c49ab-e621-4b9e-85b9-1f02daa2868b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.723466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c49ab-e621-4b9e-85b9-1f02daa2868b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.723676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c49ab-e621-4b9e-85b9-1f02daa2868b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.724574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c49ab-e621-4b9e-85b9-1f02daa2868b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:07.726257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c49ab-e621-4b9e-85b9-1f02daa2868b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:07.742526 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1359/2727] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:07.744933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:07.758963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151211cd-a81b-4f38-9ca1-fefec82da302 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:07.763905 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:07.764855 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:07.765144 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:07.766459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] PATH_INFO: `/v3/projects/354508e9f6924f8fad5ee1bcd398df0a/users/01e47747abb64bd4b17eba2dc266b31c/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:07.783721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=354508e9f6924f8fad5ee1bcd398df0a, user_id=01e47747abb64bd4b17eba2dc266b31c, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:07.786551 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:08.006322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1369/2728] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:07 2022] PUT /identity/v3/projects/354508e9f6924f8fad5ee1bcd398df0a/users/01e47747abb64bd4b17eba2dc266b31c/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:08.018837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:08.019135 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:08.019422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:08.048255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] MFA Rules not processed for user `01e47747abb64bd4b17eba2dc266b31c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:08.049601 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:08.129061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1360/2729] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:08.131815 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:08.133605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:08.194704 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783619f-bc5c-4978-b61e-31872c3e8b7f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:08.198774 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbd609-4b58-49c7-8039-9caa89608fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:08.199429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd609-4b58-49c7-8039-9caa89608fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:08.199713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd609-4b58-49c7-8039-9caa89608fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:08.199988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd609-4b58-49c7-8039-9caa89608fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:08.210249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbd609-4b58-49c7-8039-9caa89608fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:08.212162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbd609-4b58-49c7-8039-9caa89608fd9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:08.222699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1370/2730] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:14.520528 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:14.522031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:14.597622 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9d0d8-5f60-43c3-ad64-9c21258b4408 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:14.602322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:14.603216 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:14.603532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:14.603819 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] PATH_INFO: `/v3/users/f50ef229e7b847c39cf3808a3883a023` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:14.605081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50ef229e7b847c39cf3808a3883a023)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:14.607101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:14.621037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:14.865713 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50ef229e7b847c39cf3808a3883a023'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:14.874103 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50ef229e7b847c39cf3808a3883a023'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:14.888399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50ef229e7b847c39cf3808a3883a023'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:15.039840 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50ef229e7b847c39cf3808a3883a023'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:15.040540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50ef229e7b847c39cf3808a3883a023'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:15.048373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1361/2731] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:14 2022] DELETE /identity/v3/users/f50ef229e7b847c39cf3808a3883a023 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:15.053101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:15.053633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:15.053633 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:15.073555 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:15.074801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:15.151319 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1371/2732] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:15.165765 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:15.167350 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:15.235730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b32144-5ef4-489c-834c-3ce5a0160d41 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:15.239592 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:15.240208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:15.240502 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:15.240802 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:15.251809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:15.253060 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:15.266269 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1362/2733] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:15.739599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:15.741159 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:15.753818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a735d1e-4c6d-4860-bd48-7ea212721099 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:15.758676 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:15.759566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:15.760098 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:15.760394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] PATH_INFO: `/v3/projects/ecccb4837a3f428bb925dc833d765e5a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:15.761386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecccb4837a3f428bb925dc833d765e5a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:15.763389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:15.965148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecccb4837a3f428bb925dc833d765e5a', 'request_id': 'req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.091047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecccb4837a3f428bb925dc833d765e5a', 'request_id': 'req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.093847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:16.098974 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecccb4837a3f428bb925dc833d765e5a', 'request_id': 'req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.110781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] The token cache is being invalidate because project ecccb4837a3f428bb925dc833d765e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:16.111075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.111750 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.113374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1372/2734] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:15 2022] DELETE /identity/v3/projects/ecccb4837a3f428bb925dc833d765e5a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:16.117465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:16.128008 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c29e0-fcde-4523-95ab-c14e154fd53e service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:16.198971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:16.200290 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:16.200665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:16.201116 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] PATH_INFO: `/v3/users/e9a81eb7fe95465da488761b530db8d4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:16.203144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a81eb7fe95465da488761b530db8d4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:16.205805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:16.491034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:16.502459 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a81eb7fe95465da488761b530db8d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.509394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a81eb7fe95465da488761b530db8d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.522098 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a81eb7fe95465da488761b530db8d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.604691 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a81eb7fe95465da488761b530db8d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.605369 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a81eb7fe95465da488761b530db8d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:16.612104 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1363/2735] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:16 2022] DELETE /identity/v3/users/e9a81eb7fe95465da488761b530db8d4 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:16.899127 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:16.901253 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:16.933095 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad13b30-efc6-4edd-9dcf-5a74d16fb4c8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:17.019716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:17.020809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:17.021265 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:17.021558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] PATH_INFO: `/v3/users/01e47747abb64bd4b17eba2dc266b31c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:17.022943 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e47747abb64bd4b17eba2dc266b31c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:17.025056 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:17.167461 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:17.169830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:17.183598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee43713-5cc4-4a4d-a5ac-95c6187850ba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:17.189659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:17.190574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:17.190861 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:17.191146 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] PATH_INFO: `/v3/projects/05eb7bc3c799461a8a334e96d1b95008` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:17.192456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=05eb7bc3c799461a8a334e96d1b95008)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:17.194452 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:17.378791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e47747abb64bd4b17eba2dc266b31c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.386255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05eb7bc3c799461a8a334e96d1b95008', 'request_id': 'req-e274842f-8b33-4473-b75a-36452ee905ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.394772 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e47747abb64bd4b17eba2dc266b31c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.397642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05eb7bc3c799461a8a334e96d1b95008', 'request_id': 'req-e274842f-8b33-4473-b75a-36452ee905ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.495927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e47747abb64bd4b17eba2dc266b31c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.496707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e47747abb64bd4b17eba2dc266b31c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.497535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff2dfc1f-3a22-44cc-8817-3520f5f52e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e47747abb64bd4b17eba2dc266b31c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.517324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1373/2736] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:16 2022] DELETE /identity/v3/users/01e47747abb64bd4b17eba2dc266b31c => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:17.524087 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:17.524581 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:17.525111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:17.555541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:17.557644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:17.588402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05eb7bc3c799461a8a334e96d1b95008', 'request_id': 'req-e274842f-8b33-4473-b75a-36452ee905ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.601936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] The token cache is being invalidate because project 05eb7bc3c799461a8a334e96d1b95008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:17.602321 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.603118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e274842f-8b33-4473-b75a-36452ee905ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:17.604872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1364/2737] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:17 2022] DELETE /identity/v3/projects/05eb7bc3c799461a8a334e96d1b95008 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:17.609897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d02a7-a7b0-40a7-8e8e-5ea543dbd3c8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:17.616640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d02a7-a7b0-40a7-8e8e-5ea543dbd3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:17.616640 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d02a7-a7b0-40a7-8e8e-5ea543dbd3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:17.633938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a95d02a7-a7b0-40a7-8e8e-5ea543dbd3c8 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:17.635353 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a95d02a7-a7b0-40a7-8e8e-5ea543dbd3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:17.653353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1374/2738] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:17.664248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:17.665745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:17.678275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:17.742314 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfff14-fe97-4103-b0b6-ac6c726e751c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:17.743295 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1365/2739] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:17.746310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:17.747448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:17.747741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:17.748136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:17.754647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc74dc-f7dd-43c1-9a68-b527ee9b623e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:17.755040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc74dc-f7dd-43c1-9a68-b527ee9b623e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:17.755395 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc74dc-f7dd-43c1-9a68-b527ee9b623e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:17.760916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:17.762455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:17.778651 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1375/2740] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:17.929234 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-77dc74dc-f7dd-43c1-9a68-b527ee9b623e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:17.931002 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-77dc74dc-f7dd-43c1-9a68-b527ee9b623e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:17.941218 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1366/2741] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:17.945789 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:17.956096 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d514-485c-43de-a9ea-43471f6d57a7 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:17.960954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:17.961620 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:17.962166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:17.962450 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:17.963190 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:17.963500 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:17.965188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:17.980373 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1376/2742] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:56:17.985570 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc74dc-f7dd-43c1-9a68-b527ee9b623e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:17.995897 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc74dc-f7dd-43c1-9a68-b527ee9b623e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.000834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.002557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.002557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.002557 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.002981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.005106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.116149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1367/2743] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:17 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:18.120353 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.142094 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050049-40f5-467a-b429-4c30385d73a5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.213992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.215013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.215340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.215646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.216369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.219031 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.318343 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1377/2744] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:18 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:18.319534 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.331329 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9dcb-ef64-4a65-afef-d1c892534ae5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.335944 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.336902 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.337198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.337473 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.338203 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.339804 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.341213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:18.352483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1368/2745] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:18.355394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.356250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.360129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:18.367799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32546b98-0a6a-450c-9f48-f545c0cb8e7e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.371763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b3900-f5ce-43dc-8e9f-cc5ddec535cc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.377202 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.377491 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.378342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.378475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.378562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.378658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] PATH_INFO: `/v3/projects/354508e9f6924f8fad5ee1bcd398df0a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.378866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.379050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.379907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.380022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354508e9f6924f8fad5ee1bcd398df0a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.382255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.382391 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.395565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1369/2746] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:56:18.399558 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.411317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd5683-7681-4989-8b89-dddc436397cd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.416142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b9758-322b-44f2-8542-3f21c03d44c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.417086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b9758-322b-44f2-8542-3f21c03d44c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.417378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b9758-322b-44f2-8542-3f21c03d44c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.417656 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b9758-322b-44f2-8542-3f21c03d44c9 admin admin] PATH_INFO: `/v3/projects/0f75fb6528734728939e5ff79729208c/users/8e9efd617c4f4c3f9fbf3c812e511804/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.432967 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b9758-322b-44f2-8542-3f21c03d44c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f75fb6528734728939e5ff79729208c, user_id=8e9efd617c4f4c3f9fbf3c812e511804, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.435642 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b9758-322b-44f2-8542-3f21c03d44c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.539482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1370/2747] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:18 2022] PUT /identity/v3/projects/0f75fb6528734728939e5ff79729208c/users/8e9efd617c4f4c3f9fbf3c812e511804/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:18.556507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354508e9f6924f8fad5ee1bcd398df0a', 'request_id': 'req-4cf1d296-9b5b-444d-9884-4716595f3896'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:18.608449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.608885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.609044 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.636599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] MFA Rules not processed for user `8e9efd617c4f4c3f9fbf3c812e511804`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:18.638178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:18.673060 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354508e9f6924f8fad5ee1bcd398df0a', 'request_id': 'req-4cf1d296-9b5b-444d-9884-4716595f3896'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:18.680932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354508e9f6924f8fad5ee1bcd398df0a', 'request_id': 'req-4cf1d296-9b5b-444d-9884-4716595f3896'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:18.694809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] The token cache is being invalidate because project 354508e9f6924f8fad5ee1bcd398df0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:18.695233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:18.695854 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cf1d296-9b5b-444d-9884-4716595f3896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:18.701209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1378/2748] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:18 2022] DELETE /identity/v3/projects/354508e9f6924f8fad5ee1bcd398df0a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:18.703745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f84d8c-dac9-4bc0-974b-8074f8583181 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.704038 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f84d8c-dac9-4bc0-974b-8074f8583181 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.704321 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f84d8c-dac9-4bc0-974b-8074f8583181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.713232 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1371/2749] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:18.726448 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.728591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:18.729950 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b4f84d8c-dac9-4bc0-974b-8074f8583181 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:18.731787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4f84d8c-dac9-4bc0-974b-8074f8583181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:18.737523 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:18.812273 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57234017-0a10-4b7a-b695-f2fe7fbd5dfb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.816644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.817470 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.817758 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.818041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.818472 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1379/2750] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:18.820137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:18.827587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab9e19-3833-400c-9b30-f7008437bb0b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.827996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab9e19-3833-400c-9b30-f7008437bb0b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.828511 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab9e19-3833-400c-9b30-f7008437bb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.838261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.840411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.852106 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1372/2751] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:18.852492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-13ab9e19-3833-400c-9b30-f7008437bb0b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:18.853751 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13ab9e19-3833-400c-9b30-f7008437bb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:18.863489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1380/2752] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:18.867879 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.879708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0791-b6e8-4458-a4dd-45ce345eb96f service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.886031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.886754 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.887062 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.889985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.889985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:18.889985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.890575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:18.903555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1373/2753] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:56:18.908656 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab9e19-3833-400c-9b30-f7008437bb0b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:18.919073 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab9e19-3833-400c-9b30-f7008437bb0b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:18.926992 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:18.927897 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:18.928401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:18.929009 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:18.930218 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:18.933408 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:19.070743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1381/2754] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:18 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:19.075049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:19.086285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13548c62-af16-4b24-ac53-776d08cf514f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:19.128183 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:19.150261 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:19.150939 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:19.151224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:19.151504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:19.152208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:19.154192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:19.303033 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1374/2755] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:19 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:19.307335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:19.319349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992bb91-21ac-4313-982b-3508699ea737 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:19.323890 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:19.324558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:19.324874 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:19.325152 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:19.325876 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:19.328064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:19.340831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1382/2756] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:56:19.345101 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:19.355880 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf305-4f4a-462e-91b5-e4e5906687f3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:19.361055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c28ff1-f3ab-405d-ad57-0a08a3bb2607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:19.361729 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28ff1-f3ab-405d-ad57-0a08a3bb2607 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:19.362019 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28ff1-f3ab-405d-ad57-0a08a3bb2607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:19.362319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28ff1-f3ab-405d-ad57-0a08a3bb2607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:19.363050 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c28ff1-f3ab-405d-ad57-0a08a3bb2607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:19.364676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c28ff1-f3ab-405d-ad57-0a08a3bb2607 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:19.375623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1375/2757] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:19.380917 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:19.395374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdcaa1-4ed4-41f6-a551-b3ed6fb8c7a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:19.400374 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:19.401866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:19.402379 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:19.402814 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] PATH_INFO: `/v3/projects/814fb31543c7466aba93c358fd89cbfe/users/7f09369068774bc5bd38cd2d8e59ef1a/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:19.423032 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] RBAC: Authorizing `identity:create_grant(project_id=814fb31543c7466aba93c358fd89cbfe, user_id=7f09369068774bc5bd38cd2d8e59ef1a, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:19.425828 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:19.543053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1383/2758] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:19 2022] PUT /identity/v3/projects/814fb31543c7466aba93c358fd89cbfe/users/7f09369068774bc5bd38cd2d8e59ef1a/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:19.555147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6be0a-bce6-4854-85c1-00a1e75a57f0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:19.555426 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6be0a-bce6-4854-85c1-00a1e75a57f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:19.555706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6be0a-bce6-4854-85c1-00a1e75a57f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:19.590431 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bee6be0a-bce6-4854-85c1-00a1e75a57f0 None None] MFA Rules not processed for user `7f09369068774bc5bd38cd2d8e59ef1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:19.591785 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bee6be0a-bce6-4854-85c1-00a1e75a57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:19.673545 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1376/2759] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:19.681697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:19.755586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a20f6-6d06-4e8a-93b5-ad557a4b728e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:19.761847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:19.762963 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:19.763389 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:19.763801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:19.765405 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:19.767634 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:19.917835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1384/2760] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:19 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:56:19.924208 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6be0a-bce6-4854-85c1-00a1e75a57f0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:19.938616 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6be0a-bce6-4854-85c1-00a1e75a57f0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:19.999784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.000654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.000976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.001257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.001991 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.003912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:20.098176 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1377/2761] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:19 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:20.102429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:20.113756 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c2ec-4a43-4cbc-bd8a-e188a8e82a7c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:20.118777 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.119443 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.119727 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.120004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.120758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.122385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:20.133932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1385/2762] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:20.138047 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:20.149201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7979f0-9ab5-4472-9206-4d2c9aab9dd3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:20.153947 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0439b-81b4-4876-baf9-17000c3bb56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.154621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0439b-81b4-4876-baf9-17000c3bb56e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.154907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0439b-81b4-4876-baf9-17000c3bb56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.155221 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0439b-81b4-4876-baf9-17000c3bb56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.156475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0439b-81b4-4876-baf9-17000c3bb56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.158201 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0439b-81b4-4876-baf9-17000c3bb56e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:20.169504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1378/2763] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:20.176055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:20.186355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4173b4a-36b7-40c6-bb54-35b01fce8fc3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:20.191352 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.192248 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.192535 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.192847 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] PATH_INFO: `/v3/projects/44eb02d2131543d1915558f5ea607bf4/users/a850e595235a47748180ff74c9c0c32f/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.208007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44eb02d2131543d1915558f5ea607bf4, user_id=a850e595235a47748180ff74c9c0c32f, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.210737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:20.214172 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:20.342760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1386/2764] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:20 2022] PUT /identity/v3/projects/44eb02d2131543d1915558f5ea607bf4/users/a850e595235a47748180ff74c9c0c32f/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:20.409923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6fdd5-87b5-4681-8fd6-83f2cc61dbc8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.410503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6fdd5-87b5-4681-8fd6-83f2cc61dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.410503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6fdd5-87b5-4681-8fd6-83f2cc61dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.438391 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-97e6fdd5-87b5-4681-8fd6-83f2cc61dbc8 None None] MFA Rules not processed for user `a850e595235a47748180ff74c9c0c32f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:20.439623 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-97e6fdd5-87b5-4681-8fd6-83f2cc61dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:20.452286 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:20.507411 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1379/2765] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:20.511647 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:20.577899 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ca5cc-bbcf-4aef-8cda-8ef17166d4d4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:20.583483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.584225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.584544 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.584934 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.585697 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.588019 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:20.683481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1387/2766] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:20 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:20.685368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6fdd5-87b5-4681-8fd6-83f2cc61dbc8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:20.698240 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6fdd5-87b5-4681-8fd6-83f2cc61dbc8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:20.767811 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.768517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.768855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.769095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.769816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.771748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:20.845961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1380/2767] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:20 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:56:20.850074 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:20.861509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da31900-2ff2-4674-bda7-fe2d69c6fdd2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:20.866549 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.867244 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.867534 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.868001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.868732 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.870384 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:20.882629 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1388/2768] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:20.892274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:20.902956 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5b66-2e9d-46fb-a7d6-ea51e276ac52 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:20.907788 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:20.908689 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:20.909004 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:20.909278 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] PATH_INFO: `/v3/projects/1efdcff692fa459492abd5c06c2c6e76/users/b3cbc8832c3f499baf0e4e52bef334d7/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:20.928296 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efdcff692fa459492abd5c06c2c6e76, user_id=b3cbc8832c3f499baf0e4e52bef334d7, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:20.931040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:21.000368 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1381/2769] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:20 2022] PUT /identity/v3/projects/1efdcff692fa459492abd5c06c2c6e76/users/b3cbc8832c3f499baf0e4e52bef334d7/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:21.004492 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:21.015679 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8c39b-e91d-4487-9d5d-11985224f5da admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:21.073368 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34050b8-da1b-4deb-9ff1-458371daa7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:21.074050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34050b8-da1b-4deb-9ff1-458371daa7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:21.074334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34050b8-da1b-4deb-9ff1-458371daa7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:21.074612 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34050b8-da1b-4deb-9ff1-458371daa7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:21.075346 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34050b8-da1b-4deb-9ff1-458371daa7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:21.076977 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34050b8-da1b-4deb-9ff1-458371daa7e7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:21.088001 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1389/2770] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:21.092337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:21.102982 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea99c-c985-4005-ad27-14211f3b488e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:21.107580 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:21.108497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:21.108755 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:21.109067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] PATH_INFO: `/v3/projects/1efdcff692fa459492abd5c06c2c6e76/users/b3cbc8832c3f499baf0e4e52bef334d7/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:21.111224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efdcff692fa459492abd5c06c2c6e76, user_id=b3cbc8832c3f499baf0e4e52bef334d7, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:21.113883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:21.125194 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1382/2771] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:21 2022] PUT /identity/v3/projects/1efdcff692fa459492abd5c06c2c6e76/users/b3cbc8832c3f499baf0e4e52bef334d7/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:21.135851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:21.136143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:21.136422 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:21.164388 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] MFA Rules not processed for user `b3cbc8832c3f499baf0e4e52bef334d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:21.165658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:21.220396 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:21.238827 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1390/2772] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:21.251099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:21.252845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:21.313182 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81313dbb-19f7-4dbb-b710-57bff6124cfe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:21.316416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:21.317124 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:21.317413 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:21.317690 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:21.339158 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:21.380059 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:21.381614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:21.391859 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1383/2773] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:23.653119 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:23.654541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:23.668109 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc64f1-a9b6-4c79-a19e-a09f44c0c03d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:23.672888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:23.677925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:23.677925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:23.677925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:23.734070 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:23.735959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:23.745698 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1391/2774] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:27.468416 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:27.478494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a4ed3-9929-4a88-b044-2e12aae738e2 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:27.484492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dbc17-8f83-4df9-b314-736b6aa45242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:27.485715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dbc17-8f83-4df9-b314-736b6aa45242 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:27.486040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dbc17-8f83-4df9-b314-736b6aa45242 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:27.486584 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dbc17-8f83-4df9-b314-736b6aa45242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:27.498675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0dbc17-8f83-4df9-b314-736b6aa45242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:27.501356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0dbc17-8f83-4df9-b314-736b6aa45242 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:27.514102 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1384/2775] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:33.714400 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:33.715348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:33.795404 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d9ec8-d890-488f-b60b-0ef4663ea556 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:33.800463 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:33.801487 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:33.802260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:33.802260 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] PATH_INFO: `/v3/users/cfdc47394a83470b8b70ad8217408dc4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:33.803409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfdc47394a83470b8b70ad8217408dc4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:33.805610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:34.037369 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfdc47394a83470b8b70ad8217408dc4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.051552 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfdc47394a83470b8b70ad8217408dc4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.164576 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfdc47394a83470b8b70ad8217408dc4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.165258 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfdc47394a83470b8b70ad8217408dc4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.165883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfdc47394a83470b8b70ad8217408dc4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.182713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1392/2776] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:33 2022] DELETE /identity/v3/users/cfdc47394a83470b8b70ad8217408dc4 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:34.187945 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:34.188465 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:34.188591 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:34.206364 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:34.207583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:34.271443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1385/2777] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:34.281851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:34.283302 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:34.297349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:34.355409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e5fc-e992-4749-915e-ca1d04e743dd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:34.362927 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:34.366192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:34.366192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:34.366192 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:34.380454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:34.382968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:34.397599 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1393/2778] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:34.439358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:34.441959 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:34.465037 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fcb5b-6996-4e3f-9fbe-30e6cb074df7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:34.474513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:34.476137 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:34.476742 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:34.477235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] PATH_INFO: `/v3/users/7f09369068774bc5bd38cd2d8e59ef1a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:34.479768 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f09369068774bc5bd38cd2d8e59ef1a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:34.484855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:34.816068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f09369068774bc5bd38cd2d8e59ef1a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.833881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f09369068774bc5bd38cd2d8e59ef1a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.852156 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f09369068774bc5bd38cd2d8e59ef1a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.903216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:34.906444 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:34.916327 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f09369068774bc5bd38cd2d8e59ef1a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.922067 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c337c004-d070-4279-bcf7-e36b3189ae77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f09369068774bc5bd38cd2d8e59ef1a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:34.929737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582910-6671-4644-849e-ecdae9dee9ac service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:34.937867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:34.939007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1386/2779] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:34 2022] DELETE /identity/v3/users/7f09369068774bc5bd38cd2d8e59ef1a => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:34.940129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:34.940947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:34.941242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] PATH_INFO: `/v3/projects/4c9a2e91519744cd973e0f0939c92422` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:34.942719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9a2e91519744cd973e0f0939c92422)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:34.946312 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:34.954833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:34.955126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:34.955381 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:34.981466 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:34.982699 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:35.007793 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:35.066376 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1387/2780] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:35.073411 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9a2e91519744cd973e0f0939c92422', 'request_id': 'req-2a192668-a85f-44ea-8aa8-bbe0036968aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:35.081970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.083475 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:35.148856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9a2e91519744cd973e0f0939c92422', 'request_id': 'req-2a192668-a85f-44ea-8aa8-bbe0036968aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:35.152293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e6a8e-921f-47e9-90e6-34906e201e67 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.156868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9a2e91519744cd973e0f0939c92422', 'request_id': 'req-2a192668-a85f-44ea-8aa8-bbe0036968aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:35.168603 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] The token cache is being invalidate because project 4c9a2e91519744cd973e0f0939c92422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:35.168932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:35.169610 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a192668-a85f-44ea-8aa8-bbe0036968aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:35.172178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1394/2781] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:34 2022] DELETE /identity/v3/projects/4c9a2e91519744cd973e0f0939c92422 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:35.177357 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c631-10ae-47bc-be61-d36c11d8c77b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.177644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c631-10ae-47bc-be61-d36c11d8c77b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.178077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c631-10ae-47bc-be61-d36c11d8c77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.203836 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8aa0c631-10ae-47bc-be61-d36c11d8c77b None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:35.206546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8aa0c631-10ae-47bc-be61-d36c11d8c77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:35.210663 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.212210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.212499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.212795 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.214721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:35.275761 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1395/2782] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:35.284071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c04d1-243c-4a4a-a121-8deefe037a95 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.284436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c04d1-243c-4a4a-a121-8deefe037a95 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.284653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c04d1-243c-4a4a-a121-8deefe037a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.302301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.303845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:35.307908 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-802c04d1-243c-4a4a-a121-8deefe037a95 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:35.309167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-802c04d1-243c-4a4a-a121-8deefe037a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:35.315872 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1388/2783] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:35.319100 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1396/2784] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:35.323876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.337454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82492ee-7cf1-46fb-99b3-7e807dace298 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.344184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.345323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.345740 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.346231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.347238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:35.347717 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.349833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:35.369247 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1389/2785] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:56:35.370517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c04d1-243c-4a4a-a121-8deefe037a95 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.381752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c04d1-243c-4a4a-a121-8deefe037a95 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.386882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.392291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.392291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.392291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.392291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.392291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:35.530933 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1397/2786] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:35 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:56:35.532533 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.545076 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e2238-cef5-4e75-a3cf-f7f092dc7290 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.566245 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:35.660614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.661013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.661178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.661662 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.662969 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.666771 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:35.769108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1390/2787] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:35 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:56:35.775148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.790626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d904f42-837f-40b6-8599-d4f002b7c3b1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.796326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.797196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.797677 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.797956 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.798816 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.800490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:35.803196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:35.823224 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1398/2788] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:35.824673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.836938 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c678786-babb-4150-ba5e-6d10a396d369 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.842881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.843741 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.844030 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.844311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.845143 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.846871 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:35.860049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1391/2789] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:35.864923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.877207 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626fb24-4bd0-429c-aadf-ec4147f49ad0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.883642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a639561-f137-4dc8-99da-49589b9c27df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.883642 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639561-f137-4dc8-99da-49589b9c27df admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.883982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639561-f137-4dc8-99da-49589b9c27df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.883982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639561-f137-4dc8-99da-49589b9c27df admin admin] PATH_INFO: `/v3/projects/bf3f09bf74324f7ca7268feb67c046f8/users/d612283280884f119b50cceae973a6ec/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.905367 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a639561-f137-4dc8-99da-49589b9c27df admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3f09bf74324f7ca7268feb67c046f8, user_id=d612283280884f119b50cceae973a6ec, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.910315 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a639561-f137-4dc8-99da-49589b9c27df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:35.929609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:35.929609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:35.945784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62db33-65a1-4011-8488-6b611112f3c5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:35.951215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:35.952259 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:35.952543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:35.952853 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] PATH_INFO: `/v3/projects/814fb31543c7466aba93c358fd89cbfe` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:35.954387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=814fb31543c7466aba93c358fd89cbfe)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:35.956565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:36.071566 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1399/2790] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:35 2022] PUT /identity/v3/projects/bf3f09bf74324f7ca7268feb67c046f8/users/d612283280884f119b50cceae973a6ec/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:36.077716 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:36.077971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:36.078219 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:36.097263 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:36.098586 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:36.118598 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814fb31543c7466aba93c358fd89cbfe', 'request_id': 'req-6ce24d32-1235-4556-bc41-b386c8c24c13'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.130149 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814fb31543c7466aba93c358fd89cbfe', 'request_id': 'req-6ce24d32-1235-4556-bc41-b386c8c24c13'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.167664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1400/2791] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:36.178275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:36.179720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:36.228236 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814fb31543c7466aba93c358fd89cbfe', 'request_id': 'req-6ce24d32-1235-4556-bc41-b386c8c24c13'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.246928 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] The token cache is being invalidate because project 814fb31543c7466aba93c358fd89cbfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:36.247421 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.249335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.252392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1392/2792] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:35 2022] DELETE /identity/v3/projects/814fb31543c7466aba93c358fd89cbfe => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:36.256517 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:36.258090 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c3104-02be-4062-9830-fa015e686858 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:36.258324 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:36.328558 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:36.329264 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:36.329550 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:36.329830 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:36.331893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:36.353589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24d32-1235-4556-bc41-b386c8c24c13 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:36.359751 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:36.360653 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:36.360971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:36.361250 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] PATH_INFO: `/v3/users/a850e595235a47748180ff74c9c0c32f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:36.362504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a850e595235a47748180ff74c9c0c32f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:36.364482 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:36.385745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:36.392184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:36.412560 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:36.414052 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:36.426071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1401/2793] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:56:36.605654 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a850e595235a47748180ff74c9c0c32f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.620560 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a850e595235a47748180ff74c9c0c32f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.649664 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a850e595235a47748180ff74c9c0c32f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.815432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a850e595235a47748180ff74c9c0c32f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.815649 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a850e595235a47748180ff74c9c0c32f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:36.827963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1393/2794] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:36 2022] DELETE /identity/v3/users/a850e595235a47748180ff74c9c0c32f => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:37.364604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:37.378694 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e7e7b-ccc3-4f5f-900a-c1415c9d0bf5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:37.495431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:37.497496 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:37.497986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:37.499048 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] PATH_INFO: `/v3/projects/44eb02d2131543d1915558f5ea607bf4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:37.501460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44eb02d2131543d1915558f5ea607bf4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:37.505387 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:37.645583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44eb02d2131543d1915558f5ea607bf4', 'request_id': 'req-446dbf5d-7e07-43a8-9efc-93d502103f62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:37.766241 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44eb02d2131543d1915558f5ea607bf4', 'request_id': 'req-446dbf5d-7e07-43a8-9efc-93d502103f62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:37.775249 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44eb02d2131543d1915558f5ea607bf4', 'request_id': 'req-446dbf5d-7e07-43a8-9efc-93d502103f62'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:37.790868 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] The token cache is being invalidate because project 44eb02d2131543d1915558f5ea607bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:37.791216 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:37.792453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446dbf5d-7e07-43a8-9efc-93d502103f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:37.794155 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1402/2795] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:37 2022] DELETE /identity/v3/projects/44eb02d2131543d1915558f5ea607bf4 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:37.799700 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:37.802394 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:37.828686 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1428c-d29b-4420-8361-86beef854b7f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:37.921198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:37.922814 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:37.923708 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:37.924198 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] PATH_INFO: `/v3/users/b3cbc8832c3f499baf0e4e52bef334d7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:37.926632 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3cbc8832c3f499baf0e4e52bef334d7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:37.930961 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:38.260091 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3cbc8832c3f499baf0e4e52bef334d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:38.276483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3cbc8832c3f499baf0e4e52bef334d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:38.293651 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3cbc8832c3f499baf0e4e52bef334d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:38.357524 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:38.358669 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:38.359116 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:38.371678 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3cbc8832c3f499baf0e4e52bef334d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:38.372862 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3cbc8832c3f499baf0e4e52bef334d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:38.386007 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1394/2796] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:37 2022] DELETE /identity/v3/users/b3cbc8832c3f499baf0e4e52bef334d7 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:38.408901 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] MFA Rules not processed for user `d612283280884f119b50cceae973a6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:38.412435 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:38.550139 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:38.556173 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1403/2797] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:56:38.574328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:38.576458 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:38.613498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:38.660173 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1a5d3-387e-4869-bdeb-7cd305a161b3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:38.664408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ae938-f57c-4c14-b94a-e71d2b6a92d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:38.665370 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ae938-f57c-4c14-b94a-e71d2b6a92d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:38.665727 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ae938-f57c-4c14-b94a-e71d2b6a92d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:38.666378 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ae938-f57c-4c14-b94a-e71d2b6a92d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:38.678068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ae938-f57c-4c14-b94a-e71d2b6a92d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:38.680075 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ae938-f57c-4c14-b94a-e71d2b6a92d0 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:38.692071 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1395/2798] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:39.130168 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:39.132702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:39.243918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd22df-b94a-4c7f-a5a7-d2950985033d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:39.253808 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:39.255394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:39.255990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:39.256465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] PATH_INFO: `/v3/projects/1efdcff692fa459492abd5c06c2c6e76` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:39.258737 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1efdcff692fa459492abd5c06c2c6e76)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:39.262588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:39.454361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1efdcff692fa459492abd5c06c2c6e76', 'request_id': 'req-fa228296-c04a-4331-ba21-b1b1653b7e83'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:39.542884 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1efdcff692fa459492abd5c06c2c6e76', 'request_id': 'req-fa228296-c04a-4331-ba21-b1b1653b7e83'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:39.552347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1efdcff692fa459492abd5c06c2c6e76', 'request_id': 'req-fa228296-c04a-4331-ba21-b1b1653b7e83'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:39.567728 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] The token cache is being invalidate because project 1efdcff692fa459492abd5c06c2c6e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:39.568053 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:39.568747 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa228296-c04a-4331-ba21-b1b1653b7e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:39.570465 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1404/2799] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:39 2022] DELETE /identity/v3/projects/1efdcff692fa459492abd5c06c2c6e76 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:56:39.584277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869d8ac-cb4d-4a96-9696-62e8552783b1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:39.589042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869d8ac-cb4d-4a96-9696-62e8552783b1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:39.589042 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869d8ac-cb4d-4a96-9696-62e8552783b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:39.610565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e869d8ac-cb4d-4a96-9696-62e8552783b1 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:39.612607 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e869d8ac-cb4d-4a96-9696-62e8552783b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:39.718784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1396/2800] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:39.728660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbffbca-ef3b-4d77-9f81-5eabb1521eff None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:39.729022 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbffbca-ef3b-4d77-9f81-5eabb1521eff None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:39.729304 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbffbca-ef3b-4d77-9f81-5eabb1521eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:39.751664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5fbffbca-ef3b-4d77-9f81-5eabb1521eff None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:39.753066 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fbffbca-ef3b-4d77-9f81-5eabb1521eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:39.761513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1405/2801] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:39.766971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869d8ac-cb4d-4a96-9696-62e8552783b1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:39.780260 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869d8ac-cb4d-4a96-9696-62e8552783b1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:39.786446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:39.787402 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:39.789629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:39.789629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:39.789629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:39.789629 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:39.791454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:39.806606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1397/2802] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:56:39.812046 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbffbca-ef3b-4d77-9f81-5eabb1521eff None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:39.823646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbffbca-ef3b-4d77-9f81-5eabb1521eff None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:39.833448 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:39.834664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:39.835105 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:39.835652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:39.837276 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:39.843446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:39.963050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1406/2803] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:39 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:39.968494 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:39.979385 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf85fd3-a088-4212-91d6-e2140abb5936 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:40.048943 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:40.050068 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:40.050565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:40.051024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:40.051962 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:40.055108 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:40.184936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1398/2804] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:39 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:40.191080 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:40.205402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783994d6-cc0e-45dd-aee2-5ecaf2a21c0e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:40.210821 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:40.211984 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:40.212436 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:40.212875 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:40.213529 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:40.215239 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:40.234850 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1407/2805] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:40.240018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:40.253540 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3c65e-35c8-499d-9c77-305c5dc8f7d4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:40.258627 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c187b-3772-40a5-a3fb-f1e0dadb8f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:40.259396 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c187b-3772-40a5-a3fb-f1e0dadb8f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:40.259739 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c187b-3772-40a5-a3fb-f1e0dadb8f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:40.260024 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c187b-3772-40a5-a3fb-f1e0dadb8f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:40.260876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c187b-3772-40a5-a3fb-f1e0dadb8f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:40.262562 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c187b-3772-40a5-a3fb-f1e0dadb8f74 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:40.275341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1399/2806] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:56:40.281043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:40.298085 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620997f-de3b-4475-8f6a-ff7ce08cca0f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:40.305795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:40.307344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:40.307848 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:40.308317 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] PATH_INFO: `/v3/projects/70e0bef9b6f2444d87159f3cf7f5ca4b/users/10a6d2c58822423291263604c0540422/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:40.335767 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e0bef9b6f2444d87159f3cf7f5ca4b, user_id=10a6d2c58822423291263604c0540422, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:40.340429 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:40.412705 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1408/2807] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:40 2022] PUT /identity/v3/projects/70e0bef9b6f2444d87159f3cf7f5ca4b/users/10a6d2c58822423291263604c0540422/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:40.425121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24940f5f-1209-4666-8da4-439895066926 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:40.425504 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24940f5f-1209-4666-8da4-439895066926 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:40.425796 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24940f5f-1209-4666-8da4-439895066926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:40.457745 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-24940f5f-1209-4666-8da4-439895066926 None None] MFA Rules not processed for user `10a6d2c58822423291263604c0540422`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:40.458920 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24940f5f-1209-4666-8da4-439895066926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:40.471747 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:40.535675 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1400/2808] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:40.548763 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:40.550717 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:40.584928 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:40.618858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e8985-36f6-41af-94e8-e68a2610b946 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:40.622690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:40.623351 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:40.624336 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:40.624604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:40.634691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:40.638877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:40.647510 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1409/2809] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:41.867833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d265d-00f9-46a3-a203-a54e2e8ea114 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:41.868483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d265d-00f9-46a3-a203-a54e2e8ea114 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:41.868483 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d265d-00f9-46a3-a203-a54e2e8ea114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:41.889728 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5a5d265d-00f9-46a3-a203-a54e2e8ea114 None None] MFA Rules not processed for user `d612283280884f119b50cceae973a6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:41.891038 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5a5d265d-00f9-46a3-a203-a54e2e8ea114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:41.979816 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1401/2810] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:47.586996 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:47.587858 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:47.665446 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3534ae-c561-4be6-a90a-ac625ab0ba1f service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:47.671464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:47.672427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:47.672715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:47.673039 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] PATH_INFO: `/v3/users/8e9efd617c4f4c3f9fbf3c812e511804` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:47.674307 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e9efd617c4f4c3f9fbf3c812e511804)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:47.676289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:47.923623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e9efd617c4f4c3f9fbf3c812e511804'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:47.944657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e9efd617c4f4c3f9fbf3c812e511804'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:48.051233 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e9efd617c4f4c3f9fbf3c812e511804'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:48.051873 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e9efd617c4f4c3f9fbf3c812e511804'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:48.052509 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e9efd617c4f4c3f9fbf3c812e511804'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:48.068844 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1410/2811] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:47 2022] DELETE /identity/v3/users/8e9efd617c4f4c3f9fbf3c812e511804 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:48.073676 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:48.074506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:48.074506 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:48.098220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:48.099508 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:48.203307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1402/2812] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:48.237781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:48.239757 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:48.334459 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:48.341225 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ecb3d-d314-4bc0-810b-73a31cc3e87e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:48.345877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c2519-30a4-49a8-b0b1-055d58b2ec06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:48.348167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c2519-30a4-49a8-b0b1-055d58b2ec06 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:48.348167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c2519-30a4-49a8-b0b1-055d58b2ec06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:48.348167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c2519-30a4-49a8-b0b1-055d58b2ec06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:48.359781 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c2519-30a4-49a8-b0b1-055d58b2ec06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:48.361590 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c2519-30a4-49a8-b0b1-055d58b2ec06 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:48.384254 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1411/2813] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:49.145126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:49.146834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:49.174451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1027e-7948-4646-9dc0-09aa32560d24 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:49.174451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:49.174451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:49.174451 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:49.174794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] PATH_INFO: `/v3/projects/0f75fb6528734728939e5ff79729208c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:49.174794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f75fb6528734728939e5ff79729208c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:49.174794 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:49.346813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f75fb6528734728939e5ff79729208c', 'request_id': 'req-d6b64469-2094-4f4b-a080-c3fa274d8c71'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:49.352948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f75fb6528734728939e5ff79729208c', 'request_id': 'req-d6b64469-2094-4f4b-a080-c3fa274d8c71'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:49.354644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:49.456341 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f75fb6528734728939e5ff79729208c', 'request_id': 'req-d6b64469-2094-4f4b-a080-c3fa274d8c71'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:49.475297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] The token cache is being invalidate because project 0f75fb6528734728939e5ff79729208c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:49.475297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:49.475297 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d6b64469-2094-4f4b-a080-c3fa274d8c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:49.476987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1403/2814] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:49 2022] DELETE /identity/v3/projects/0f75fb6528734728939e5ff79729208c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:49.485427 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c278894-2376-4a19-9c47-e102cd9f26df None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:49.486055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c278894-2376-4a19-9c47-e102cd9f26df None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:49.486420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c278894-2376-4a19-9c47-e102cd9f26df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:49.518546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4c278894-2376-4a19-9c47-e102cd9f26df None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:49.520637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c278894-2376-4a19-9c47-e102cd9f26df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:49.605466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1412/2815] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:49.613792 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b514bb9-0e91-4beb-b5b3-3559a14a0f52 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:49.614189 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b514bb9-0e91-4beb-b5b3-3559a14a0f52 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:49.614348 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b514bb9-0e91-4beb-b5b3-3559a14a0f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:49.633760 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7b514bb9-0e91-4beb-b5b3-3559a14a0f52 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:49.634978 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7b514bb9-0e91-4beb-b5b3-3559a14a0f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:49.646805 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1404/2816] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:49.649982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c278894-2376-4a19-9c47-e102cd9f26df None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:49.660386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c278894-2376-4a19-9c47-e102cd9f26df None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:49.665519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:49.666169 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:49.666454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:49.666714 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:49.667488 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:49.667802 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:49.669449 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:49.681683 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1413/2817] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:56:49.687280 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b514bb9-0e91-4beb-b5b3-3559a14a0f52 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:49.697020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b514bb9-0e91-4beb-b5b3-3559a14a0f52 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:49.702806 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:49.703585 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:49.703866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:49.704138 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:49.704907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:49.707049 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:49.807813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1405/2818] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:49 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:56:49.812752 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:49.824310 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e166-5db3-48f8-8966-cf01a2158707 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:49.893238 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:49.893923 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:49.894268 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:49.894568 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:49.895270 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:49.897242 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:50.011179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1414/2819] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:49 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:50.016055 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:50.029166 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5190-7262-4d61-ba44-81f54707f4b8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:50.034891 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:50.035572 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:50.035865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:50.036301 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:50.036985 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:50.039041 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:50.054040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1406/2820] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:50.055143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:50.066383 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6208399-5a72-4619-992f-c805e9e0bd50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:50.071385 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260b9f8-5bf9-4299-8a91-f83307c774a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:50.072091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260b9f8-5bf9-4299-8a91-f83307c774a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:50.072348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260b9f8-5bf9-4299-8a91-f83307c774a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:50.072626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260b9f8-5bf9-4299-8a91-f83307c774a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:50.073394 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260b9f8-5bf9-4299-8a91-f83307c774a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:50.075020 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260b9f8-5bf9-4299-8a91-f83307c774a6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:50.085860 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1415/2821] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:50.090665 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:50.101698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dec2a-11c9-48d5-a8db-f247c47590b6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:50.106258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:50.107142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:50.107442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:50.108337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] PATH_INFO: `/v3/projects/6ccc8499e2e24dfa8bd8eb8ea1978ba7/users/bc605fb8be754d14b21de88330efe013/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:50.124356 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ccc8499e2e24dfa8bd8eb8ea1978ba7, user_id=bc605fb8be754d14b21de88330efe013, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:50.127089 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:50.205762 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1407/2822] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:50 2022] PUT /identity/v3/projects/6ccc8499e2e24dfa8bd8eb8ea1978ba7/users/bc605fb8be754d14b21de88330efe013/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:50.216733 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df613b1d-1808-43e0-8998-272651fb7028 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:50.217741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df613b1d-1808-43e0-8998-272651fb7028 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:50.218432 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df613b1d-1808-43e0-8998-272651fb7028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:50.249016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-df613b1d-1808-43e0-8998-272651fb7028 None None] MFA Rules not processed for user `bc605fb8be754d14b21de88330efe013`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:50.250289 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-df613b1d-1808-43e0-8998-272651fb7028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:50.296886 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:50.318342 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1416/2823] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:50.329284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:50.331107 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:50.391442 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7a98-21fe-40bb-a9df-5c1c3fd019a1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:50.395210 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:50.395876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:50.396167 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:50.396688 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:50.406503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:50.407963 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:50.417244 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1408/2824] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:52.240883 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d059-aeb9-4f67-abb0-9e1d11156b44 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:52.241485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d059-aeb9-4f67-abb0-9e1d11156b44 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:52.241485 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d059-aeb9-4f67-abb0-9e1d11156b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:52.264123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2ee1d059-aeb9-4f67-abb0-9e1d11156b44 None None] MFA Rules not processed for user `bc605fb8be754d14b21de88330efe013`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:52.268153 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ee1d059-aeb9-4f67-abb0-9e1d11156b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:52.274092 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1417/2825] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:53.226858 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:53.228477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:53.305021 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0fc4c-2c12-4f17-b4ef-98184d63b09d service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:53.309881 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:53.311178 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:53.312214 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:53.312595 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] PATH_INFO: `/v3/users/10a6d2c58822423291263604c0540422` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:53.313994 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a6d2c58822423291263604c0540422)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:53.315992 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:53.343118 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:DELETE=7,keystone:SELECT=90 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:53.572748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a6d2c58822423291263604c0540422'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:53.579732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a6d2c58822423291263604c0540422'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:53.591809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a6d2c58822423291263604c0540422'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:53.776215 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a6d2c58822423291263604c0540422'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:53.776876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a6d2c58822423291263604c0540422'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:53.784179 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1409/2826] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:56:53 2022] DELETE /identity/v3/users/10a6d2c58822423291263604c0540422 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:53.789099 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:53.789580 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:53.789979 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:53.808453 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:53.809707 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:53.872166 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1418/2827] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:53.884113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:53.888599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:53.954422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2ff8f-77cc-4250-97ee-69d37c60eadf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:53.959293 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47981798-07a9-4cd9-b80b-7fb955b59d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:53.960092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47981798-07a9-4cd9-b80b-7fb955b59d69 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:53.960373 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47981798-07a9-4cd9-b80b-7fb955b59d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:53.960685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47981798-07a9-4cd9-b80b-7fb955b59d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:53.971121 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47981798-07a9-4cd9-b80b-7fb955b59d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:53.972569 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47981798-07a9-4cd9-b80b-7fb955b59d69 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:53.983292 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1410/2828] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:56:54.493489 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:54.494990 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:54.509013 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e2a9b-c7d6-48aa-bd60-c1e920c57bc4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:54.514925 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:54.515892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:54.522982 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:54.523134 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] PATH_INFO: `/v3/projects/70e0bef9b6f2444d87159f3cf7f5ca4b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:54.535696 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e0bef9b6f2444d87159f3cf7f5ca4b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:54.537826 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:54.704759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e0bef9b6f2444d87159f3cf7f5ca4b', 'request_id': 'req-4a02da59-657f-4e52-8f05-9e4926c0d6bb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:54.970965 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e0bef9b6f2444d87159f3cf7f5ca4b', 'request_id': 'req-4a02da59-657f-4e52-8f05-9e4926c0d6bb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:54.978617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e0bef9b6f2444d87159f3cf7f5ca4b', 'request_id': 'req-4a02da59-657f-4e52-8f05-9e4926c0d6bb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:54.991337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] The token cache is being invalidate because project 70e0bef9b6f2444d87159f3cf7f5ca4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:56:54.991688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:54.992366 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a02da59-657f-4e52-8f05-9e4926c0d6bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:56:54.999555 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eefb1-62c9-4abb-9116-1c346cd9b7a6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.007414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eefb1-62c9-4abb-9116-1c346cd9b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.007414 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eefb1-62c9-4abb-9116-1c346cd9b7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.007882 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1419/2829] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:56:54 2022] DELETE /identity/v3/projects/70e0bef9b6f2444d87159f3cf7f5ca4b => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:55.020809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-069eefb1-62c9-4abb-9116-1c346cd9b7a6 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:55.022361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-069eefb1-62c9-4abb-9116-1c346cd9b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:55.097444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1411/2830] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:55.106196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff520-356b-4f7f-9d61-e58da6f4f5bc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.106493 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff520-356b-4f7f-9d61-e58da6f4f5bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.106776 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff520-356b-4f7f-9d61-e58da6f4f5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.126111 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-058ff520-356b-4f7f-9d61-e58da6f4f5bc None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:55.127347 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-058ff520-356b-4f7f-9d61-e58da6f4f5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:55.135759 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1420/2831] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:55.140672 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eefb1-62c9-4abb-9116-1c346cd9b7a6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:55.151524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eefb1-62c9-4abb-9116-1c346cd9b7a6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:55.157759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:55.158715 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.159018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.159294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.160043 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:56:55.160359 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:55.162040 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:55.182274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1412/2832] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:56:55.187569 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff520-356b-4f7f-9d61-e58da6f4f5bc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:55.200675 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff520-356b-4f7f-9d61-e58da6f4f5bc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:55.206213 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:55.207226 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.207519 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.207954 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.208660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:55.210659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:55.430382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1421/2833] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:56:55 2022] POST /identity/v3/projects => generated 381 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:55.433926 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:55.445164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a59c7-c357-4aef-98a2-a568fde4ebbb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:55.478003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:55.508120 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:55.508828 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.509117 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.509399 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.510139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:55.512095 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:55.673764 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1413/2834] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:56:55 2022] POST /identity/v3/users => generated 291 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:56:55.677878 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:55.689746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffdda3-c882-4ba2-a826-17679ed2e9bb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:55.694454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:55.700999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.700999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.700999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.700999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:55.700999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:55.711415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1422/2835] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:55.715258 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:55.726161 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f56d6-99b3-4531-9b20-41c66078eb50 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:55.731456 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da587ba3-977b-4e56-8938-d722aea781b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:55.732119 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da587ba3-977b-4e56-8938-d722aea781b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.733018 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da587ba3-977b-4e56-8938-d722aea781b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.733305 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da587ba3-977b-4e56-8938-d722aea781b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.734058 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da587ba3-977b-4e56-8938-d722aea781b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:55.735694 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da587ba3-977b-4e56-8938-d722aea781b9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:55.746958 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1414/2836] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:56:55.752326 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:55.762746 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a28de-9abb-4598-889d-1523a95ba7d1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:55.768859 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:55.769699 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.769986 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.770279 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] PATH_INFO: `/v3/projects/abc1cd8a20854cb0ac847cf30c1343eb/users/5275b73fe98746879d090ab7713afd1e/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.788455 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc1cd8a20854cb0ac847cf30c1343eb, user_id=5275b73fe98746879d090ab7713afd1e, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:55.791186 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:55.902894 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:56:55.909245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1423/2837] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:56:55 2022] PUT /identity/v3/projects/abc1cd8a20854cb0ac847cf30c1343eb/users/5275b73fe98746879d090ab7713afd1e/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:56:55.917821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550234d9-27f2-45a3-a4ad-6e145495d50d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:55.917821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550234d9-27f2-45a3-a4ad-6e145495d50d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:55.917821 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550234d9-27f2-45a3-a4ad-6e145495d50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:55.938319 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-550234d9-27f2-45a3-a4ad-6e145495d50d None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:56:55.939544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-550234d9-27f2-45a3-a4ad-6e145495d50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:56.000850 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1415/2838] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:56:56.010856 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:56:56.012895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:56:56.080189 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755984c5-788d-47d6-94fb-3af3573be611 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:56:56.083969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:56:56.084857 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:56:56.085150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:56:56.085431 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:56:56.096390 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:56:56.097935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:56:56.109209 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1424/2839] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:04.699022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1faa-91dd-41c2-be45-3e418d7273b6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:04.699446 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1faa-91dd-41c2-be45-3e418d7273b6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:04.699548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1faa-91dd-41c2-be45-3e418d7273b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:04.743126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9a0a1faa-91dd-41c2-be45-3e418d7273b6 None None] MFA Rules not processed for user `5275b73fe98746879d090ab7713afd1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:04.749382 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a0a1faa-91dd-41c2-be45-3e418d7273b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:04.845392 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1416/2840] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:57:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:57:04.850267 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:04.862514 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c50b9-b2db-42b8-87d0-2062226b352e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:04.868541 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:04.869362 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:04.869867 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:04.870198 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:04.870893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:04.873071 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:05.025980 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1425/2841] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:57:04 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:57:05.031554 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a1faa-91dd-41c2-be45-3e418d7273b6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:05.043544 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a1faa-91dd-41c2-be45-3e418d7273b6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:05.095225 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:05.118339 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:05.118930 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:05.119220 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:05.119685 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:05.120393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:05.122410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:05.229074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1417/2842] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:57:05 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:57:05.233916 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:05.246322 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9155-2882-4849-baa4-819d6ad72d38 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:05.255745 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:05.256764 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:05.257337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:05.257787 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:05.259623 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:05.263050 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:05.278084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1426/2843] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:57:05.283072 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:05.294344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b5cbc-b6dd-4f66-ae75-e4988cb7dd93 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:05.299867 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:05.301621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:05.301621 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:05.301955 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] PATH_INFO: `/v3/projects/a42c425f46f34931a82458c93e39ff4b/users/bf4eec7935a447f5ba7426599b86e3a8/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:05.330122 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42c425f46f34931a82458c93e39ff4b, user_id=bf4eec7935a447f5ba7426599b86e3a8, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:05.333847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:05.425237 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1418/2844] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:57:05 2022] PUT /identity/v3/projects/a42c425f46f34931a82458c93e39ff4b/users/bf4eec7935a447f5ba7426599b86e3a8/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:57:05.429920 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:05.445188 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f6ed2-5525-4996-a4d8-c6f9d9d1250e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:05.524913 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b31061-857c-480f-8210-8eabbcf0d956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:05.525636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b31061-857c-480f-8210-8eabbcf0d956 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:05.525918 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b31061-857c-480f-8210-8eabbcf0d956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:05.526196 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b31061-857c-480f-8210-8eabbcf0d956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:05.526949 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b31061-857c-480f-8210-8eabbcf0d956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:05.528619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b31061-857c-480f-8210-8eabbcf0d956 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:05.540507 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1427/2845] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:57:05.545979 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:05.557112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277e097-ab37-41a9-afc1-a233f0eef1aa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:05.563422 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:05.564535 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:05.564999 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:05.565333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] PATH_INFO: `/v3/projects/a42c425f46f34931a82458c93e39ff4b/users/bf4eec7935a447f5ba7426599b86e3a8/roles/fa0d2e27b7af41708eee88eac77dbebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:05.568907 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42c425f46f34931a82458c93e39ff4b, user_id=bf4eec7935a447f5ba7426599b86e3a8, role_id=fa0d2e27b7af41708eee88eac77dbebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:05.574499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:05.591809 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1419/2846] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:57:05 2022] PUT /identity/v3/projects/a42c425f46f34931a82458c93e39ff4b/users/bf4eec7935a447f5ba7426599b86e3a8/roles/fa0d2e27b7af41708eee88eac77dbebf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:57:12.906091 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23db0c7-6cfb-4985-b752-cd754f8cc8e7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:12.907033 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23db0c7-6cfb-4985-b752-cd754f8cc8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:12.907478 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23db0c7-6cfb-4985-b752-cd754f8cc8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:12.972736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c23db0c7-6cfb-4985-b752-cd754f8cc8e7 None None] MFA Rules not processed for user `bf4eec7935a447f5ba7426599b86e3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:12.977010 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c23db0c7-6cfb-4985-b752-cd754f8cc8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:13.098271 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:13.126217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1428/2847] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:57:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:57:13.141550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:13.143579 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:13.215719 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5bd1c-c718-461e-adba-d95dd5ba5598 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:13.220393 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:13.223730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:13.223730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:13.223730 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:13.282968 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:13.284432 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:13.295457 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1420/2848] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:57:13.580964 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a44e7-87b8-4d16-b241-b404b8ab95e5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:13.581306 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a44e7-87b8-4d16-b241-b404b8ab95e5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:13.581600 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a44e7-87b8-4d16-b241-b404b8ab95e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:13.604101 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c40a44e7-87b8-4d16-b241-b404b8ab95e5 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:13.605460 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c40a44e7-87b8-4d16-b241-b404b8ab95e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:13.672128 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1429/2849] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:57:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:57:13.856923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:13.856923 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:13.907721 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:13.930776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a55dc9-0098-4237-b499-dc56b6231add service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:13.938192 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:13.938611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:13.939020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:13.939575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:13.951507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:13.953242 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:13.962157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1421/2850] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:57:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:57:14.564862 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6c3a-ea86-49c5-9b87-0bab8190c4c7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:14.565734 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6c3a-ea86-49c5-9b87-0bab8190c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:14.566177 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6c3a-ea86-49c5-9b87-0bab8190c4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:14.594888 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a8ed6c3a-ea86-49c5-9b87-0bab8190c4c7 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:14.596440 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a8ed6c3a-ea86-49c5-9b87-0bab8190c4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:14.611007 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1430/2851] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:57:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:57:14.622774 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:14.635174 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6c9f-80b0-4657-9611-931f002a6cb0 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:14.641518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:14.642430 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:14.642874 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:14.643300 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:14.655337 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:14.657868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:14.668388 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1422/2852] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:57:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:57:16.060340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a4795-b2f0-4dc8-9b9d-1f36d466bc9f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:16.060340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a4795-b2f0-4dc8-9b9d-1f36d466bc9f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:16.060340 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a4795-b2f0-4dc8-9b9d-1f36d466bc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:16.078604 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-194a4795-b2f0-4dc8-9b9d-1f36d466bc9f None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:16.080167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-194a4795-b2f0-4dc8-9b9d-1f36d466bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:16.113893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1431/2853] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 18 10:57:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:57:16.123235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:16.147020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a64abe-81a0-42ca-b34e-4f58eb38d0b8 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:16.165551 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:16.166836 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:16.168336 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:16.168900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:16.188113 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:16.190499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:16.210263 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1423/2854] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:57:17.415106 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fdf70-dcd1-44de-88c9-b92342108304 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:17.416205 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fdf70-dcd1-44de-88c9-b92342108304 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:17.416501 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fdf70-dcd1-44de-88c9-b92342108304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:17.442344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0c8fdf70-dcd1-44de-88c9-b92342108304 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:17.442344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c8fdf70-dcd1-44de-88c9-b92342108304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:17.453349 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1432/2855] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:57:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:57:17.497289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:17.497289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bace04-da9f-4f7e-8bc9-8822b1ce618d service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:17.497289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:17.497289 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:17.498284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:17.498284 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:17.506489 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:17.510125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:17.530246 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1424/2856] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:57:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:57:18.832337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:18.833337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:18.833619 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:18.858892 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:18.860425 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:18.868587 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1433/2857] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 18 10:57:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:57:18.887387 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:18.898693 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b14eb8-094f-49f6-9be1-2a2e7280f26e service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:18.905077 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf69871-439b-46bf-b161-ba0b642bdb7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:18.906036 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf69871-439b-46bf-b161-ba0b642bdb7f service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:18.906480 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf69871-439b-46bf-b161-ba0b642bdb7f service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:18.906909 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf69871-439b-46bf-b161-ba0b642bdb7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:18.918147 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf69871-439b-46bf-b161-ba0b642bdb7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:18.921159 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf69871-439b-46bf-b161-ba0b642bdb7f service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:18.931499 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1425/2858] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:57:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:57:22.646820 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:22.649481 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:22.762420 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fe4e2-e850-4a59-9176-0e17ca129e14 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:22.768401 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:22.769946 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:22.770402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:22.770835 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] PATH_INFO: `/v3/users/bc605fb8be754d14b21de88330efe013` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:22.772375 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc605fb8be754d14b21de88330efe013)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:22.774617 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:23.018148 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc605fb8be754d14b21de88330efe013'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:23.045466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc605fb8be754d14b21de88330efe013'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:23.049914 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:23.130409 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc605fb8be754d14b21de88330efe013'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:23.131129 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc605fb8be754d14b21de88330efe013'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:23.131804 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc605fb8be754d14b21de88330efe013'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:23.152932 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1434/2859] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:57:22 2022] DELETE /identity/v3/users/bc605fb8be754d14b21de88330efe013 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:57:23.157644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:23.159140 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:23.160659 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:23.183503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:23.187673 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:23.289772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1426/2860] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:57:23.302266 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:23.303211 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:23.382228 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a094b-bfaf-4a04-81f6-a7627997b03e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:23.386921 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff4a6e-9848-45b0-8db2-0f11c11b86e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:23.389123 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4a6e-9848-45b0-8db2-0f11c11b86e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:23.390061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4a6e-9848-45b0-8db2-0f11c11b86e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:23.390061 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4a6e-9848-45b0-8db2-0f11c11b86e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:23.401433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff4a6e-9848-45b0-8db2-0f11c11b86e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:23.403768 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff4a6e-9848-45b0-8db2-0f11c11b86e2 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:23.424866 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1435/2861] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:23.727028 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:23.728807 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:23.747919 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf7e6e-0a3e-477d-95cd-4a437605886e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:23.753358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:23.754157 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:23.754439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:23.754763 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] PATH_INFO: `/v3/projects/6ccc8499e2e24dfa8bd8eb8ea1978ba7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:23.756449 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ccc8499e2e24dfa8bd8eb8ea1978ba7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:23.758723 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:23.966303 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ccc8499e2e24dfa8bd8eb8ea1978ba7', 'request_id': 'req-5f7c0749-0b70-425e-a728-d3f078cde568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:23.982032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ccc8499e2e24dfa8bd8eb8ea1978ba7', 'request_id': 'req-5f7c0749-0b70-425e-a728-d3f078cde568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:24.062230 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ccc8499e2e24dfa8bd8eb8ea1978ba7', 'request_id': 'req-5f7c0749-0b70-425e-a728-d3f078cde568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:24.084081 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] The token cache is being invalidate because project 6ccc8499e2e24dfa8bd8eb8ea1978ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:57:24.084503 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:24.085358 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:24.104813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1427/2862] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:57:23 2022] DELETE /identity/v3/projects/6ccc8499e2e24dfa8bd8eb8ea1978ba7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:57:26.303998 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:26.304851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:26.304851 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:26.323294 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] MFA Rules not processed for user `5275b73fe98746879d090ab7713afd1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:26.324439 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:26.388466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1436/2863] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:57:26.429082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:26.430776 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:26.441255 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:26.524086 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0749-0b70-425e-a728-d3f078cde568 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:26.530772 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138e41c-ec01-49f1-b5fa-f55422ab3d81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80535e7da62e4790931fd74000653411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:26.531443 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e41c-ec01-49f1-b5fa-f55422ab3d81 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:26.531732 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e41c-ec01-49f1-b5fa-f55422ab3d81 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:26.532009 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e41c-ec01-49f1-b5fa-f55422ab3d81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:26.534885 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f138e41c-ec01-49f1-b5fa-f55422ab3d81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:26.630854 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138e41c-ec01-49f1-b5fa-f55422ab3d81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:26.632363 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138e41c-ec01-49f1-b5fa-f55422ab3d81 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:26.646078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1428/2864] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:27.461399 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:27.464034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:27.539546 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40029d3-5ab2-4082-b580-42cb6f607776 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:27.544187 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:27.545201 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:27.545583 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:27.545938 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:27.548086 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:27.566895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:27.568819 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:27.585137 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1437/2865] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:31.305514 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:31.306323 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:31.306507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:31.326492 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:57:31.327833 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:31.396142 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1429/2866] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 18 10:57:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:57:31.410044 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:31.411556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:31.468026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:31.500665 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64374a-3846-44e4-a319-34fe8881ba8e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:31.507344 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:31.508217 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:31.508502 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:31.508837 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:31.525415 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:31.528337 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:31.548232 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1438/2867] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:57:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:57:41.384799 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=85 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:41.667324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=15 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:52.823397 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:52.825692 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:52.860462 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0b299-1ab2-4306-8b26-0afd9088c611 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:52.873185 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:52.874546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:52.875003 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:52.875320 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:52.877589 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:52.894017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:52.897507 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:52.920903 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1430/2868] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:59.583721 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:59.585877 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:59.603457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfea9c-73b3-4f31-a4f2-8ec63cd9ae71 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:59.609657 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be951d2b-d1b9-4460-939c-a9ebd9389c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:59.627636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be951d2b-d1b9-4460-939c-a9ebd9389c92 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:59.627636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be951d2b-d1b9-4460-939c-a9ebd9389c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:59.627636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be951d2b-d1b9-4460-939c-a9ebd9389c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:59.627636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be951d2b-d1b9-4460-939c-a9ebd9389c92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:59.627636 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be951d2b-d1b9-4460-939c-a9ebd9389c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:59.629240 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be951d2b-d1b9-4460-939c-a9ebd9389c92 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:59.642795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1439/2869] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:58:01.351748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:01.353778 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:01.372020 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d786f-d715-4ded-9ad3-e71f2d988ffe service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:01.378169 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:01.378837 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:01.379125 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:01.379406 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:01.382031 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:01.397453 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:01.399151 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:01.413876 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1431/2870] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:58:05.170122 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7d3e3-6a04-4999-b8d0-b2a38eb13aea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:05.170702 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7d3e3-6a04-4999-b8d0-b2a38eb13aea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:05.170906 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7d3e3-6a04-4999-b8d0-b2a38eb13aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:05.190526 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6df7d3e3-6a04-4999-b8d0-b2a38eb13aea None None] MFA Rules not processed for user `d612283280884f119b50cceae973a6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:58:05.191758 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6df7d3e3-6a04-4999-b8d0-b2a38eb13aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:05.256075 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1440/2871] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:58:06.092521 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:06.093609 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:06.201241 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8b592-ef80-449d-be79-b23f3684e5d2 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:06.206911 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:06.207614 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:06.207905 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:06.208184 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:06.210498 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:06.225477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:06.227111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:06.239718 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1432/2872] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:58:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:58:09.345688 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7d3e3-6a04-4999-b8d0-b2a38eb13aea None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:09.359324 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7d3e3-6a04-4999-b8d0-b2a38eb13aea None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:09.364255 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:09.364961 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:09.365275 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:09.365701 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:09.368968 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:09.453120 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:09.454736 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:09.467935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1441/2873] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:58:12.634975 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:12.650226 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd3e35-73b9-4df7-9d7a-51486ed101db service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:12.655575 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:12.656257 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:12.656546 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:12.656855 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:12.659034 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:12.767476 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:12.769543 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:12.784759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1433/2874] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:58:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:58:17.630708 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:17.670627 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e5e81-2686-42a8-952c-8a0bf8c66692 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:17.693947 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec899ae414744dde9ec9b76aac1ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:17.695562 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:17.696350 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:17.709931 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:17.713434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:17.751940 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:17.753988 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:17.796483 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1442/2875] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:58:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:58:20.190333 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:20.192307 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:20.212469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d05ac-5f44-4bfe-b0fb-d61863df186a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:20.231066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:20.231066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:20.231066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:20.231066 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] PATH_INFO: `/v3/users/eb483b1aee944da8910dd7468a0ef4ac` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:20.231826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb483b1aee944da8910dd7468a0ef4ac)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:20.231826 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:20.632566 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb483b1aee944da8910dd7468a0ef4ac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:20.645322 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:58:20.649317 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb483b1aee944da8910dd7468a0ef4ac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:20.666497 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb483b1aee944da8910dd7468a0ef4ac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:20.786229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb483b1aee944da8910dd7468a0ef4ac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:20.786866 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb483b1aee944da8910dd7468a0ef4ac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:20.796051 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1434/2876] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:58:20 2022] DELETE /identity/v3/users/eb483b1aee944da8910dd7468a0ef4ac => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:58:21.219274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:21.234791 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848609-bdb7-44de-9670-fbe55d76d360 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:21.327658 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:21.328670 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:21.328999 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:21.329274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] PATH_INFO: `/v3/projects/17f436162f3a4bd4b3a94f584e7d8782` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:21.330654 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f436162f3a4bd4b3a94f584e7d8782)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:21.332652 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:21.358594 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:58:21.525280 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f436162f3a4bd4b3a94f584e7d8782', 'request_id': 'req-62006e0c-26ef-441e-8b3c-620236e2d4cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:21.661000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f436162f3a4bd4b3a94f584e7d8782', 'request_id': 'req-62006e0c-26ef-441e-8b3c-620236e2d4cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:21.673397 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f436162f3a4bd4b3a94f584e7d8782', 'request_id': 'req-62006e0c-26ef-441e-8b3c-620236e2d4cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:21.692605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] The token cache is being invalidate because project 17f436162f3a4bd4b3a94f584e7d8782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:58:21.696466 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:21.697849 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:21.702495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1443/2877] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:58:21 2022] DELETE /identity/v3/projects/17f436162f3a4bd4b3a94f584e7d8782 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:58:21.707112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:21.709743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:21.727454 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a1948-2698-49c0-a6e2-91db1291b654 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:21.826238 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:21.827574 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:21.828674 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:21.829082 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] PATH_INFO: `/v3/users/2e37b061ecdb4577b13e3cf2615d29d4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:21.830893 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e37b061ecdb4577b13e3cf2615d29d4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:21.833243 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:22.195571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e37b061ecdb4577b13e3cf2615d29d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:22.203725 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e37b061ecdb4577b13e3cf2615d29d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:22.219311 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e37b061ecdb4577b13e3cf2615d29d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:22.375518 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e37b061ecdb4577b13e3cf2615d29d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:22.376335 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e37b061ecdb4577b13e3cf2615d29d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:22.383801 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1435/2878] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:58:21 2022] DELETE /identity/v3/users/2e37b061ecdb4577b13e3cf2615d29d4 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:58:22.696715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:58:22.698952 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:58:22.713115 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006e0c-26ef-441e-8b3c-620236e2d4cf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:58:22.776885 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:58:22.778000 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:58:22.778297 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:58:22.778659 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] PATH_INFO: `/v3/projects/78719e43f24b499e8647f50db6771cba` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:58:22.780386 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78719e43f24b499e8647f50db6771cba)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:58:22.782454 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:58:22.997713 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78719e43f24b499e8647f50db6771cba', 'request_id': 'req-cdab0daf-d0a4-4d69-a559-832090e88b02'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:23.123593 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78719e43f24b499e8647f50db6771cba', 'request_id': 'req-cdab0daf-d0a4-4d69-a559-832090e88b02'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:23.130831 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78719e43f24b499e8647f50db6771cba', 'request_id': 'req-cdab0daf-d0a4-4d69-a559-832090e88b02'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:23.143037 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] The token cache is being invalidate because project 78719e43f24b499e8647f50db6771cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:58:23.143335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:23.144016 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:58:23.145653 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1444/2879] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:58:22 2022] DELETE /identity/v3/projects/78719e43f24b499e8647f50db6771cba => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:58:32.379661 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=63,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:58:33.139843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=69,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:18.613933 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:18.615477 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:18.694282 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ee6f9-3511-49b9-a565-91ebf7067400 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:18.757606 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:18.758248 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:18.758646 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:18.759542 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:18.761647 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:18.773386 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:18.776209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:18.787013 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1436/2880] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:59:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:59:18.807556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:18.811464 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:18.886508 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab0daf-d0a4-4d69-a559-832090e88b02 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:18.891291 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:18.892167 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:18.892447 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:18.892726 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] PATH_INFO: `/v3/users/d612283280884f119b50cceae973a6ec` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:18.893935 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d612283280884f119b50cceae973a6ec)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:18.895869 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:19.107084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd612283280884f119b50cceae973a6ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.119795 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd612283280884f119b50cceae973a6ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.201083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd612283280884f119b50cceae973a6ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.201770 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd612283280884f119b50cceae973a6ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.202473 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa957bb-0c6d-4c0f-8171-d788fb1ecc86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd612283280884f119b50cceae973a6ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.216971 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1445/2881] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:59:18 2022] DELETE /identity/v3/users/d612283280884f119b50cceae973a6ec => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:19.493644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:19.508274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6e0cc-3eee-451a-8468-fc1d238b9b81 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:19.567648 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:19.568524 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:19.568830 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:19.569111 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] PATH_INFO: `/v3/projects/bf3f09bf74324f7ca7268feb67c046f8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:19.570302 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3f09bf74324f7ca7268feb67c046f8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:19.572314 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:19.713139 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:19.723064 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3f09bf74324f7ca7268feb67c046f8', 'request_id': 'req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.730532 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3f09bf74324f7ca7268feb67c046f8', 'request_id': 'req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.825900 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3f09bf74324f7ca7268feb67c046f8', 'request_id': 'req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.838527 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] The token cache is being invalidate because project bf3f09bf74324f7ca7268feb67c046f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:59:19.838831 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.839513 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7eb2fb7d-be35-47f3-ae7c-75bd2835d5a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:19.841126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1437/2882] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:59:19 2022] DELETE /identity/v3/projects/bf3f09bf74324f7ca7268feb67c046f8 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:19.846084 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de99e1-cb19-4d48-9fca-d3e440e61c27 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:19.846660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de99e1-cb19-4d48-9fca-d3e440e61c27 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:19.846660 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de99e1-cb19-4d48-9fca-d3e440e61c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:19.864907 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-05de99e1-cb19-4d48-9fca-d3e440e61c27 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:59:19.866138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-05de99e1-cb19-4d48-9fca-d3e440e61c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:19.929893 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1446/2883] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:59:19.938784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f00ce-4f17-4cb3-b2dd-b4d3380d0a61 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:19.938784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f00ce-4f17-4cb3-b2dd-b4d3380d0a61 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:19.938784 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f00ce-4f17-4cb3-b2dd-b4d3380d0a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:19.959404 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-540f00ce-4f17-4cb3-b2dd-b4d3380d0a61 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:59:19.960617 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-540f00ce-4f17-4cb3-b2dd-b4d3380d0a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:19.970231 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1438/2884] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:59:19.974664 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de99e1-cb19-4d48-9fca-d3e440e61c27 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:19.985690 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de99e1-cb19-4d48-9fca-d3e440e61c27 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:19.990382 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:19.991043 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:19.991325 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:19.991602 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:19.992335 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:59:19.992637 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:19.994801 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:20.007800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1447/2885] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 18 10:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:59:20.013912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f00ce-4f17-4cb3-b2dd-b4d3380d0a61 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:20.023825 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f00ce-4f17-4cb3-b2dd-b4d3380d0a61 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:20.030559 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:20.031328 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:20.031610 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:20.031892 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:20.032597 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:20.034550 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:20.141285 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1439/2886] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 18 10:59:20 2022] POST /identity/v3/projects => generated 391 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:59:20.146410 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:20.156843 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdc441-f094-4625-91af-542c98f2dc97 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:20.160639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:20.217381 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760a472-4763-48c4-a508-323657377374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:20.218077 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760a472-4763-48c4-a508-323657377374 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:20.218360 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760a472-4763-48c4-a508-323657377374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:20.218639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760a472-4763-48c4-a508-323657377374 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:20.219451 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760a472-4763-48c4-a508-323657377374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:20.221469 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760a472-4763-48c4-a508-323657377374 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:20.346433 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1448/2887] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 18 10:59:20 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:59:20.350724 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:20.361469 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1a2eb-0758-425f-9dab-87f06f190726 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:20.366224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:20.366906 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:20.367196 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:20.367474 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:20.368957 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:20.370577 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:20.382409 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1440/2888] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:59:20.398735 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760a472-4763-48c4-a508-323657377374 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:20.411434 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760a472-4763-48c4-a508-323657377374 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:20.417034 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10954d1c-f78c-482d-b225-0b8ed12f4079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:20.417741 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954d1c-f78c-482d-b225-0b8ed12f4079 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:20.418029 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954d1c-f78c-482d-b225-0b8ed12f4079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:20.418334 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954d1c-f78c-482d-b225-0b8ed12f4079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:20.419083 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10954d1c-f78c-482d-b225-0b8ed12f4079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:20.420720 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10954d1c-f78c-482d-b225-0b8ed12f4079 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:20.433625 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1449/2889] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 18 10:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:59:20.438896 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:20.457233 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c2b5b-cd66-435f-af62-6889ea2826ff admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:20.465783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:20.467424 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:20.467932 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:20.468224 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] PATH_INFO: `/v3/projects/bd909fa646d440a0aecadc3fdf3fec51/users/4f44ff2ca071455cac300240ea115e98/roles/9cc91942b0de4b1bb86179386ef67e09` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:20.496274 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd909fa646d440a0aecadc3fdf3fec51, user_id=4f44ff2ca071455cac300240ea115e98, role_id=9cc91942b0de4b1bb86179386ef67e09)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:20.501847 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:20.619995 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1441/2890] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 18 10:59:20 2022] PUT /identity/v3/projects/bd909fa646d440a0aecadc3fdf3fec51/users/4f44ff2ca071455cac300240ea115e98/roles/9cc91942b0de4b1bb86179386ef67e09 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:20.631136 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daf55a-6d6d-4fb0-9d75-c609a2358b03 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:20.635755 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daf55a-6d6d-4fb0-9d75-c609a2358b03 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:20.635755 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daf55a-6d6d-4fb0-9d75-c609a2358b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:20.665611 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a3daf55a-6d6d-4fb0-9d75-c609a2358b03 None None] MFA Rules not processed for user `4f44ff2ca071455cac300240ea115e98`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:59:20.666800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a3daf55a-6d6d-4fb0-9d75-c609a2358b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:20.738939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1450/2891] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:59:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:59:20.751583 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:20.753354 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:20.821783 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec53286-81c7-4f70-bb59-5c0be4f0d7bb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:20.826571 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62e462a1b125428f9893459d5d965ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:20.827439 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:20.827726 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:20.828001 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:20.839706 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:20.841277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:20.851970 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1442/2892] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:59:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:59:30.735709 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:30.847078 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:42.308818 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8948-0226-4bcc-a09f-4a6c10a55e5e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:42.309355 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8948-0226-4bcc-a09f-4a6c10a55e5e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:42.309691 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8948-0226-4bcc-a09f-4a6c10a55e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:42.366959 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-069e8948-0226-4bcc-a09f-4a6c10a55e5e None None] MFA Rules not processed for user `41cc665cbec74e06bfa7acf97f968e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:59:42.368715 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-069e8948-0226-4bcc-a09f-4a6c10a55e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:42.440969 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1451/2893] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 18 10:59:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:59:42.485417 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:42.486868 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:42.590548 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff03d4e-1f19-47f5-9863-b2bdbe6e1cf1 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:42.598605 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51664310-1d4c-4a2a-b868-723e6acdd8d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '642a42337be14f10b410903df57f7527', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:42.599277 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51664310-1d4c-4a2a-b868-723e6acdd8d8 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:42.599565 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51664310-1d4c-4a2a-b868-723e6acdd8d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:42.599845 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51664310-1d4c-4a2a-b868-723e6acdd8d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:42.609639 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51664310-1d4c-4a2a-b868-723e6acdd8d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:42.611379 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51664310-1d4c-4a2a-b868-723e6acdd8d8 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:42.622144 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1443/2894] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 18 10:59:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:59:42.643068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8948-0226-4bcc-a09f-4a6c10a55e5e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:42.717936 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8948-0226-4bcc-a09f-4a6c10a55e5e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:42.724626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:42.724626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:42.724626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:42.724626 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] PATH_INFO: `/v3/users/4f44ff2ca071455cac300240ea115e98` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:42.725987 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f44ff2ca071455cac300240ea115e98)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:42.728520 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:43.007047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f44ff2ca071455cac300240ea115e98'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:43.020588 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f44ff2ca071455cac300240ea115e98'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:43.126182 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f44ff2ca071455cac300240ea115e98'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:43.126760 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f44ff2ca071455cac300240ea115e98'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:43.127423 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f44ff2ca071455cac300240ea115e98'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:43.130004 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:43.141743 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1452/2895] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:59:42 2022] DELETE /identity/v3/users/4f44ff2ca071455cac300240ea115e98 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:43.146834 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:43.147105 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:43.147408 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:43.169759 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] MFA Rules not processed for user `bf6162d0fbf040d78eb20f2b6ea7ba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:59:43.170987 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:43.234017 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1444/2896] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 18 10:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:59:43.245064 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:43.246644 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:43.312179 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58775-e51b-4c2b-acef-6db462f195fa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:43.315895 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a48b0fdacb2c43649916304e1237f39c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:43.316832 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:43.317124 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:43.317402 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:43.329782 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:43.331939 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:43.345274 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1453/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 10:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:59:43.874578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:43.874578 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:43.885022 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6ac87-679b-4d5b-98a9-688f0406a798 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:43.890644 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:43.891604 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:43.891971 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:43.892345 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] PATH_INFO: `/v3/projects/bd909fa646d440a0aecadc3fdf3fec51` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:43.893912 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd909fa646d440a0aecadc3fdf3fec51)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:43.896092 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:44.038787 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd909fa646d440a0aecadc3fdf3fec51', 'request_id': 'req-534f6006-4968-4b5d-999c-6b8b30362e6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:44.047209 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd909fa646d440a0aecadc3fdf3fec51', 'request_id': 'req-534f6006-4968-4b5d-999c-6b8b30362e6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:44.122599 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd909fa646d440a0aecadc3fdf3fec51', 'request_id': 'req-534f6006-4968-4b5d-999c-6b8b30362e6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:44.128948 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:44.137061 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] The token cache is being invalidate because project bd909fa646d440a0aecadc3fdf3fec51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:59:44.137444 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:44.138355 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:44.139936 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1445/2898] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:59:43 2022] DELETE /identity/v3/projects/bd909fa646d440a0aecadc3fdf3fec51 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:52.010926 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:52.012605 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:52.032513 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b980590-90c1-4a28-a5c2-fa968d4c4000 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:52.098578 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:52.099361 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:52.099646 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:52.099919 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] PATH_INFO: `/v3/users/5275b73fe98746879d090ab7713afd1e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:52.101175 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5275b73fe98746879d090ab7713afd1e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:52.104026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:52.400245 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5275b73fe98746879d090ab7713afd1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:52.412556 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5275b73fe98746879d090ab7713afd1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:52.505348 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5275b73fe98746879d090ab7713afd1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:52.506047 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5275b73fe98746879d090ab7713afd1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:52.506667 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5275b73fe98746879d090ab7713afd1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:52.521178 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1454/2899] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:59:52 2022] DELETE /identity/v3/users/5275b73fe98746879d090ab7713afd1e => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:52.814608 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:52.816112 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:52.832423 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6006-4968-4b5d-999c-6b8b30362e6c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:52.909883 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:52.910748 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:52.911032 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:52.911309 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] PATH_INFO: `/v3/projects/abc1cd8a20854cb0ac847cf30c1343eb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:52.912813 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=abc1cd8a20854cb0ac847cf30c1343eb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:52.915405 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:53.052743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abc1cd8a20854cb0ac847cf30c1343eb', 'request_id': 'req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.062375 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abc1cd8a20854cb0ac847cf30c1343eb', 'request_id': 'req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.143228 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abc1cd8a20854cb0ac847cf30c1343eb', 'request_id': 'req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.155844 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] The token cache is being invalidate because project abc1cd8a20854cb0ac847cf30c1343eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:59:53.156126 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.157884 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.158400 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1446/2900] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:59:52 2022] DELETE /identity/v3/projects/abc1cd8a20854cb0ac847cf30c1343eb => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:53.162517 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:53.164024 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:53.177809 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246f619-aa4a-4da8-92fa-05bf02e4dd0e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:53.187150 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:53.241018 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:53.242495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:53.242495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:53.242495 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] PATH_INFO: `/v3/users/bf4eec7935a447f5ba7426599b86e3a8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:53.244068 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4eec7935a447f5ba7426599b86e3a8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:53.246598 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:53.487019 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4eec7935a447f5ba7426599b86e3a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.500131 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4eec7935a447f5ba7426599b86e3a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.611490 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4eec7935a447f5ba7426599b86e3a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.612138 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4eec7935a447f5ba7426599b86e3a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.612765 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4eec7935a447f5ba7426599b86e3a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:53.627800 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1455/2901] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 18 10:59:53 2022] DELETE /identity/v3/users/bf4eec7935a447f5ba7426599b86e3a8 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:59:53.907660 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:59:53.909924 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:59:53.924306 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed1f7a-c1e4-47c2-bcdb-22d63bbad7ba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:59:53.989523 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6162d0fbf040d78eb20f2b6ea7ba47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '233b60ec452b417c902a9645e05969db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:59:53.990410 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:59:53.990698 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:59:53.990976 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] PATH_INFO: `/v3/projects/a42c425f46f34931a82458c93e39ff4b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:59:53.992213 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42c425f46f34931a82458c93e39ff4b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:59:53.995235 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:59:54.132865 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:59:54.151863 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42c425f46f34931a82458c93e39ff4b', 'request_id': 'req-b100d889-d07b-4deb-8a9c-5166e3b90fa9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:54.162014 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42c425f46f34931a82458c93e39ff4b', 'request_id': 'req-b100d889-d07b-4deb-8a9c-5166e3b90fa9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:54.247981 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42c425f46f34931a82458c93e39ff4b', 'request_id': 'req-b100d889-d07b-4deb-8a9c-5166e3b90fa9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:54.261365 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] The token cache is being invalidate because project a42c425f46f34931a82458c93e39ff4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:59:54.265099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:54.265099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:59:54.265099 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1447/2902] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 18 10:59:53 2022] DELETE /identity/v3/projects/a42c425f46f34931a82458c93e39ff4b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 11:00:03.624026 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=47,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 11:00:04.323187 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 11:00:08.191162 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 11:00:08.192592 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 11:00:08.269457 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427b40c-63c8-49dd-976c-22d1174bd5c0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 11:00:08.336055 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980df4e-fab6-4264-b5c9-644e3c37bccb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 11:00:08.336719 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980df4e-fab6-4264-b5c9-644e3c37bccb service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 11:00:08.337027 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980df4e-fab6-4264-b5c9-644e3c37bccb service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 11:00:08.337303 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980df4e-fab6-4264-b5c9-644e3c37bccb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 11:00:08.339143 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b980df4e-fab6-4264-b5c9-644e3c37bccb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 11:00:08.350991 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980df4e-fab6-4264-b5c9-644e3c37bccb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 11:00:08.352639 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980df4e-fab6-4264-b5c9-644e3c37bccb service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 11:00:08.363477 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1456/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 11:00:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 11:00:18.358887 dv-cinder-849498-1-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=58 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 11:05:10.210611 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 11:05:10.212164 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 11:05:10.328361 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d889-d07b-4deb-8a9c-5166e3b90fa9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 11:05:10.337035 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fd40d-df1e-44be-9ad3-34e7656522fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8defe1e0d31249aa8bb27610229781b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2509fcdb62684f97a2842ce68f7081fc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 11:05:10.338229 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fd40d-df1e-44be-9ad3-34e7656522fc service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 11:05:10.338743 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fd40d-df1e-44be-9ad3-34e7656522fc service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 11:05:10.339294 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fd40d-df1e-44be-9ad3-34e7656522fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 11:05:10.356074 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fd40d-df1e-44be-9ad3-34e7656522fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 11:05:10.359344 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fd40d-df1e-44be-9ad3-34e7656522fc service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 11:05:10.380916 dv-cinder-849498-1-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1448/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 18 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)